.. ## .. ## Copyright (c) Lawrence Livermore National Security, LLC and other .. ## RAJA Project Developers. See top-level LICENSE and COPYRIGHT .. ## files for dates and other details. No copyright assignment is required .. ## to contribute to RAJA. .. ## .. ## SPDX-License-Identifier: (BSD-3-Clause) .. ## .. _feat-message-label: =============================== Messages =============================== ``RAJA::messages`` provides a portable interface and type-safe way to store messages that can be handled at a later time. In this context, a message will store function arguments and handled typically means passed to a function. For example, one use case could be logging error messages from the GPU to file. In this case, arguments from the GPU can be stored and passed to function that prints to a file on the CPU. .. warning:: This capability is new. We would like users to try it out and give feedback to improve it. -------------------------- How to manage messages? -------------------------- All messages are handled via the ``message_manager``, which is responsible for storing callbacks and a list of messages. For the purposes of ``RAJA::messages``, a single message can be thought of: * ``MsgHeader``: helper data internal to ``RAJA`` * ``MsgArgs``: a tuple of arguments needed to pass to the function To create the ``message_manager``: .. literalinclude:: ../../../../examples/messages-forall.cpp :start-after: _raja_msg_manager_start :end-before: _raja_msg_manager_end :language: C++ ``buf_sz`` is the size of the buffer that stores messages. ``res_host`` is the resource of the execution policy, which determines the memory space in which messages will be stored. For GPU resources, for example, this is ``PINNED`` memory. Subscribing callbacks ^^^^^^^^^^^^^^^^^^^^^ To create a specific message type, callbacks must subscribe first. This can be done in two ways: * ``subscribe(Callable)``: Subscribing with just a callable will create a new type of message with the type depending on the parameters. * ``subscribe(msg_queue_id, Callable)``: Subscribing with ``msg_queue_id`` and a callable will append the new callback to the already existing callback list. As an example for subscribing with both methods: .. literalinclude:: ../../../../examples/messages-forall.cpp :start-after: _raja_msg_subscribe_start :end-before: _raja_msg_subscribe_end :language: C++ Unsubscribing callbacks ^^^^^^^^^^^^^^^^^^^^^^^ If a particular callback no longer needs to be subscribed to a message type, then the callback can be unsubcribed. This can be achieved in three ways: * ``unsubscribe(msg_queue_id, Callable)``: Looks for a specific callback that is subscribed to a particular message. If the callback is subscribed, remove from callback list. Otherwise, throws an exception. * ``unsubscribe_all(msg_queue_id)``: Removes all callbacks subscribed to a particular message * ``unsubscribe_all()``: Removes all callbacks from map. * ``erase_all(msg_queue_id)``: Erases all callbacks and the map entry. An example for unsubscribing a callback: .. literalinclude:: ../../../../examples/messages-forall.cpp :start-after: _raja_msg_unsubscribe_start :end-before: _raja_msg_unsubscribe_end :language: C++ Publishing messages ^^^^^^^^^^^^^^^^^^^ Messages can be published/stored in a ``MessageQueue``. These are non-owning adapters to the ``MessageBus``, which is responsible for storing all messages. The ``MessageQueue`` will contain additional type information as well as the ``msg_queue_id``. A queue is created once a callback is subscribed to a new message type. Since ``MessageQueue`` is non-owning, these can be copied. Here is how the ``MessageQueue`` can be used to publish messages: .. literalinclude:: ../../../../examples/messages-forall.cpp :start-after: _raja_msg_k2_start :end-before: _raja_msg_k2_end :language: C++ Handling messages ^^^^^^^^^^^^^^^^^ Lastly, there needs to be a way to direct messages to the corresponding callback(s). This is handled with the ``message_manager``, which forces a synchronize on the resource provided. .. literalinclude:: ../../../../examples/messages-forall.cpp :start-after: _raja_msg_wait_start :end-before: _raja_msg_wait_end :language: C++ Handling messages with a GPU ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Here is a complete example using the ``RAJA::messages`` with a GPU kernel. .. literalinclude:: ../../../../examples/messages-forall.cpp :start-after: _raja_msg_gpu1_start :end-before: _raja_msg_gpu1_end :language: C++ .. note:: In this example, ``gpu_policy`` depends on the build (i.e., CUDA, HIP), ``res`` is the default resource for ``gpu_policy``, and ``d_*`` arrays are allocated for the device. These are removed above from the example to just show the ``RAJA::messages`` interface. Handling messages across multiple streams ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Here is a complete example using the ``RAJA::messages`` with multiple resources. .. literalinclude:: ../../../../examples/messages-forall.cpp :start-after: _raja_msg_gpu2_start :end-before: _raja_msg_gpu2_end :language: C++ .. note:: In this example, ``res_gpu1`` and ``res_gpu2`` depend on the build (i.e., CUDA, HIP), ``EXEC_POLICY`` the exeuction policy for the loops (also depends on the build), and ``d_*`` arrays are allocated for the device while ``h_*`` are allocated for the host. These are removed from the example above to just show the ``RAJA::messages`` interface. Message queue policies ^^^^^^^^^^^^^^^^^^^^^^ Message queues can support various policies depending on the requirements of that queue, such as the number of producers/consumers or the type of atomic operations. ======================= ============================ Message queue Policies Brief description ======================= ============================ spsc Supports a single producer, single consumer; i.e., no atomic operations mpsc Supports multiple producers, a single consumer; i.e., requires atomic operations. Automatically determines which atomic operations to use. ======================= ============================ .. note:: Producers and consumers can not operate at the same time Building and running the example ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The example ``examples/messages-forall.cpp`` is built from the RAJA source tree when: * ``ENABLE_EXAMPLES=On`` To run the example: .. code-block:: bash ./bin/messages-forall This example will show how callbacks can be subscribed to various types of messages as well as how to publish messages on multiple different platforms. For the purposes of this example, output for messages will be printed using ``std::cout``. -------------------------- Application considerations -------------------------- There are several things to consider when using ``RAJA::messages`` in an application. * The ``MessageQueue`` with the correct argument types is created when a callback subscribes. Certain patterns will cause this storage to slowly grow overtime. For example, creating a new ``MessageQueue`` every function call within a loop. Therefore, applications that use this pattern will want to unsubscribe at some point to avoid running out of memory. * Upon creation of the ``MessageManager``, the ``MessageBus`` will be allocated with some size. This can be resized; however, resizing will force a synchronize and will loss any messages currently stored. Also, by default, the allocation is done through the resource, which can be less performant depending on the resource. * Since the ``MessageQueue`` is a fixed size, there is a chance of lossing messages. The ``try_post_message`` function will return a ``boolean``. This will be ``true`` if the message is successfully added to the queue; otherwise, this is ``false``. Allocators ^^^^^^^^^^ During the creation of the ``MessageManager``, a custom allocator needs to be provided. This allocator should follow the C++ standard requirements for an allocator and will be used to allocate/deallocate the message bus. The message bus will always read messages on the host to forward the message to the respect callback(s). However, depending on the build and where messages are stored, messages may be written on either the host or the device. Therefore, in cases where messages are stored on the device, the allocator needs to allocate memory that is accessible on both the host and device. .. note:: When using the move assignment operator, the ``MessageManager`` assumes that there are no currently pending messages.