🍪
cookielau
  • Introduction
  • Machine Learning
    • Distributed
      • Bookmarks
    • NLP
      • Transformers
    • MLC
      • Tensor Program Abstraction
      • End-to-End Module Execution
  • Framework
    • PyTorch
      • Bookmarks
      • Model
      • Shared
      • Miscellaneous
    • Tensorflow
      • Bookmarks
      • Model
      • Shared
      • Miscellaneous
    • CUDA
      • Bookmarks
    • DeepSpeed
    • Bagua
      • Model
      • Optimizer
    • Others
      • Bookmarks
  • About Me
    • 2022-04-28
  • Random Thoughts
  • Archives
    • CPP
      • Bookmarks
      • Container
      • Algorithm
      • FILE CONTROL
      • Virtual Table
      • Assembly
      • Key Words
      • Problems
      • Others
    • JAVA
      • String Container
      • Maps
    • PYTHON
      • Bookmarks
      • Python Tools
        • Batch Rename
        • Combine Excel
        • Excel Oprations
        • Read Write Excel
        • Rotate PDF
      • Library
        • Pandas Notes
        • Numpy Notes
        • Json Notes
      • Spider
        • Selenium Install
        • Selenium Locating
        • Selenium Errors
        • Selenium Basics
      • Django
        • Start Up
      • Others
    • LINUX
      • Installation
      • Cli Tools
      • WSL
      • Bugs
    • JUNIOR2
      • Economics
        • Chapter 0x01 经济管理概述
        • Chapter 0x02 微观市场机制分析
        • Chapter 0x03 生产决策与市场结构
        • Chapter 0x04 宏观经济市场分析
        • Chapter 0x05 管理的职能
        • Chapter 0x06 生产系统结构与战略
        • Chapter 0x0b 投资项目经济评价
        • Chapter 0x0f 投资项目经济评价
      • Computer Network
        • 概述
        • 分层模型
        • 物理层
        • 数据链路层
        • 网络层
        • 传输层
        • 应用层
        • HTTP(s)实验
        • [Practice]
      • Software Engineering
        • Introduction
        • Demand Analysis
        • Task Estimation
        • Presentation
      • Network Security
        • Chapter 0x01 概述
        • Chapter 0x02 密码学
        • Chapter 0x03 公钥体制
        • Chapter 0x04 消息认证
        • Chapter 0x05 密钥管理
        • Chapter 0x06 访问控制
        • Assignments
      • x86 Programming
        • Basic Knowledge
        • Program Design
        • System Interruption
        • Frequently used functions
    • MD&LaTex
      • Markdown
      • LaTex
    • NPM
      • NPM LINK
    • MyBlogs
      • 2020BUAA软工——“停下来,回头看”
      • 2020BUAA软工——“初窥构建之法”
      • 2020BUAA软工——“上手软件工程,PSP初体验!”
      • 2020BUAA软工——“深度评测官”
      • 2020BUAA软工——“并肩作战,平面交点Pro”
    • SC
      • PAC 2022
        • Lectures
      • OpenMP & MPI
        • MPI Overview
        • Message Passing Programming
        • OpenMP Overview
        • Work Sharing Directives
        • Annual Challenge
        • Future Topics in OpenMP
        • Tasks
        • OpenMP & MPI
    • Hardware
      • Nvidia GPU
        • Frequent Error
        • Memory Classification
        • CUDA_7_Streams_Simplify_Concurrency
        • Optimize_Data_Transfers_in_CUDA
        • Overlap_Data_Transfers_in_CUDA
        • Write_Flexible_Kernels_with_Grid-Stride_Loops
        • How_to_Access_Global_Memory_Efficiently
        • Using_Shared_Memory
      • Intel CPU
        • Construction
        • Optimization
        • Compilation
        • OpenMP
    • English
      • Vocab
      • Composition
    • Interview
      • Computer Network
Powered by GitBook
On this page

Was this helpful?

  1. Archives
  2. SC
  3. OpenMP & MPI

Tasks

Definition:

  • Task construct defines a section of code.

  • Inside a parallel region, a thread encountering a task construct will package up the task for execution.

  • Some thread in the parallel region will execute the task at some point in the future.

#pragma omp task [clauses]
structured-block

Data Sharing:

  • The default for tasks is usually FIRSTPRIVATE, because the task may not be executed until later (and variables may have gone out of scope).

  • Variables that are shared in all constructs starting from the innermost enclosing parallel construct are shared.

#pragma omp parallel shared(A) private(B)
{
    ...
    #pragma omp task
    {
        int C;
        compute(A, B, C); // A is shared, B is firstprivate, C is private
    }
}
  • Getting data attribute scoping right can be quite tricky, using default(none) is a good idea.

Task complete time:

  • At thread barriers, whether explicit or implicit

    • at the end of current parallel region, there is an implicit barrier to all tasks

  • At taskwait directive

    • Wait until all tasks defined in the current task have completed.

    • #pragma omp taskwait

    • Note: applies only to tasks generated in the current task, not to "descendants".

// Example of taskwait: Binary Tree Traversal

void postorder(node* p) {
    if (p->left) {
        #pragma omp task
        {
            postorder(p->left);
        }
    }
    if (p->right) {
        #pragma omp task 
        {
            postorder(p->right);
        }
    }
    #pragma omp taskwait // Parent task suspended until children tasks complete
    process(p->data);
}

Task switching:

  • Certain constructs have task scheduling points at defined locations within them

  • When a thread encounters a task scheduling point, it is allowed to suspend the current task and execute another (called task switching)

  • It can then return to the original task and resume.

omp_lock_t *lock;
for (int i = 0; i < n; i++) {
    #pragma omp task
    {
        something_useful();
        while (!omp_test_lock(lock)) {
            #pragma omp taskyield
            // to make full use of computing resources 
            // and avoid deadlock situations.
        }
        something_critical();
        omp_unset_lock(lock);
    }
}

Reference

PreviousFuture Topics in OpenMPNextOpenMP & MPI

Last updated 2 years ago

Was this helpful?

Example of taskyield from

SC15
OpenMP Tasking | SC15