Derecho  0.9
Distributed systems toolkit for RDMA
Public Member Functions | Public Attributes | Private Types | Private Member Functions | Private Attributes | Friends | List of all members
sst::SST< DerivedSST > Class Template Reference

#include <predicates.hpp>

Collaboration diagram for sst::SST< DerivedSST >:
Collaboration graph
[legend]

Public Member Functions

 SST (DerivedSST *derived_class_pointer, const SSTParams &params)
 
template<typename... Fields>
void SSTInit (Fields &... fields)
 
 ~SST ()
 Destructor for the SST object; sets thread_shutdown to true and waits for background threads to exit cleanly. More...
 
void start_predicate_evaluation ()
 Starts the predicate evaluation loop. More...
 
void sync_with_members () const
 Does a TCP sync with each member of the SST. More...
 
void sync_with_members (std::vector< uint32_t > row_indices) const
 Syncs with a subset of the members. More...
 
void freeze (int row_index)
 Marks a row as frozen, so it will no longer update, and its corresponding node will not receive writes. More...
 
unsigned int get_num_rows () const
 Returns the total number of rows in the table. More...
 
unsigned int get_local_index () const
 Gets the index of the local row in the table. More...
 
const char * getBaseAddress ()
 
void put ()
 Writes the entire local row to all remote nodes. More...
 
void put_with_completion ()
 
void put (const std::vector< uint32_t > receiver_ranks)
 Writes the entire local row to some of the remote nodes. More...
 
void put_with_completion (const std::vector< uint32_t > receiver_ranks)
 
void put (size_t offset, size_t size)
 Writes a contiguous subset of the local row to all remote nodes. More...
 
template<typename T >
void put (SSTField< T > &field)
 Writes a specific local field to all remote nodes. More...
 
template<typename T >
void put (SSTFieldVector< T > &vec_field)
 Writes a specific local vector field to all remote nodes. More...
 
template<typename T >
void put (SSTFieldVector< T > &vec_field, std::size_t index)
 Writes only a single element of a vector field to all remote nodes. More...
 
template<typename T >
void put (const std::vector< uint32_t > receiver_ranks, SSTFieldVector< T > &vec_field, std::size_t index)
 Writes only a single element of a vector field to only some of the remote nodes. More...
 
void put_with_completion (size_t offset, size_t size)
 
void put (const std::vector< uint32_t > receiver_ranks, size_t offset, size_t size)
 Writes a contiguous subset of the local row to some of the remote nodes. More...
 
void put_with_completion (const std::vector< uint32_t > receiver_ranks, size_t offset, size_t size)
 

Public Attributes

Predicates< DerivedSST > predicates
 

Private Types

using char_p = volatile char *
 

Private Member Functions

template<typename... Fields>
void init_SSTFields (Fields &... fields)
 
void detect ()
 This function is run in a detached background thread to detect predicate events. More...
 
void compute_rowLen (size_t &)
 
template<typename Field , typename... Fields>
void compute_rowLen (size_t &rowLen, Field &f, Fields &... rest)
 
void set_bases_and_rowLens (char_p &, const size_t)
 
template<typename Field , typename... Fields>
void set_bases_and_rowLens (char_p &base, const size_t rlen, Field &f, Fields &... rest)
 

Private Attributes

DerivedSST * derived_this
 
std::vector< std::thread > background_threads
 
std::atomic< bool > thread_shutdown
 
const uint32_t poll_cq_timeout_ms
 timeout settings for poll completion queue More...
 
volatile char * rows
 Pointer to memory where the SST rows are stored. More...
 
size_t rowLen
 Length of each row in this SST, in bytes. More...
 
const std::vector< uint32_t > & members
 List of nodes in the SST; indexes are row numbers, values are node IDs. More...
 
const unsigned int num_members
 Equal to members.size() More...
 
std::vector< uint32_t > all_indices
 
unsigned int my_index
 Index (row number) of this node in the SST. More...
 
std::map< uint32_t, int, std::greater< uint32_t > > members_by_id
 Maps node IDs to SST row indexes. More...
 
uint32_t my_node_id
 ID of this node in the system. More...
 
std::vector< bool > row_is_frozen
 Map of queue pair number to row. More...
 
int num_frozen {0}
 The number of rows that have been frozen. More...
 
failure_upcall_t failure_upcall
 The function to call when a remote node appears to have failed. More...
 
std::mutex freeze_mutex
 Mutex for failure detection and row freezing. More...
 
std::vector< std::unique_ptr< resources > > res_vec
 RDMA resources vector, one for each member. More...
 
bool thread_start
 Indicates whether the predicate evaluation thread should start after being forked in the constructor. More...
 
std::mutex thread_start_mutex
 Mutex for thread_start_cv. More...
 
std::condition_variable thread_start_cv
 Notified when the predicate evaluation thread should start. More...
 

Friends

class Predicates< DerivedSST >
 

Detailed Description

template<class DerivedSST>
class sst::SST< DerivedSST >

Definition at line 13 of file predicates.hpp.

Member Typedef Documentation

◆ char_p

template<class DerivedSST>
using sst::SST< DerivedSST >::char_p = volatile char*
private

Definition at line 389 of file sst.hpp.

Constructor & Destructor Documentation

◆ SST()

template<class DerivedSST>
sst::SST< DerivedSST >::SST ( DerivedSST< DerivedSST > *  derived_class_pointer,
const SSTParams params 
)
inline

Definition at line 227 of file sst.hpp.

◆ ~SST()

template<typename DerivedSST >
sst::SST< DerivedSST >::~SST ( )

Destructor for the SST object; sets thread_shutdown to true and waits for background threads to exit cleanly.

Definition at line 30 of file sst_impl.hpp.

Member Function Documentation

◆ compute_rowLen() [1/2]

template<class DerivedSST>
void sst::SST< DerivedSST >::compute_rowLen ( size_t &  )
inlineprivate

Definition at line 391 of file sst.hpp.

◆ compute_rowLen() [2/2]

template<class DerivedSST>
template<typename Field , typename... Fields>
void sst::SST< DerivedSST >::compute_rowLen ( size_t &  rowLen,
Field &  f,
Fields &...  rest 
)
inlineprivate

Definition at line 394 of file sst.hpp.

◆ detect()

template<typename DerivedSST >
void sst::SST< DerivedSST >::detect ( )
private

This function is run in a detached background thread to detect predicate events.

It continuously evaluates predicates one by one, and runs the trigger functions for each predicate that fires. In addition, it continuously evaluates named functions one by one, and updates the local row's observed values of those functions.

Definition at line 61 of file sst_impl.hpp.

◆ freeze()

template<typename DerivedSST >
void sst::SST< DerivedSST >::freeze ( int  row_index)

Marks a row as frozen, so it will no longer update, and its corresponding node will not receive writes.

Definition at line 247 of file sst_impl.hpp.

◆ get_local_index()

template<class DerivedSST>
unsigned int sst::SST< DerivedSST >::get_local_index ( ) const
inline

Gets the index of the local row in the table.

Definition at line 318 of file sst.hpp.

◆ get_num_rows()

template<class DerivedSST>
unsigned int sst::SST< DerivedSST >::get_num_rows ( ) const
inline

Returns the total number of rows in the table.

Definition at line 315 of file sst.hpp.

◆ getBaseAddress()

template<class DerivedSST>
const char* sst::SST< DerivedSST >::getBaseAddress ( )
inline

Definition at line 320 of file sst.hpp.

◆ init_SSTFields()

template<class DerivedSST>
template<typename... Fields>
void sst::SST< DerivedSST >::init_SSTFields ( Fields &...  fields)
inlineprivate

Definition at line 163 of file sst.hpp.

◆ put() [1/8]

template<class DerivedSST>
void sst::SST< DerivedSST >::put ( )
inline

Writes the entire local row to all remote nodes.

Definition at line 325 of file sst.hpp.

◆ put() [2/8]

template<class DerivedSST>
void sst::SST< DerivedSST >::put ( const std::vector< uint32_t >  receiver_ranks)
inline

Writes the entire local row to some of the remote nodes.

Definition at line 334 of file sst.hpp.

◆ put() [3/8]

template<class DerivedSST>
void sst::SST< DerivedSST >::put ( size_t  offset,
size_t  size 
)
inline

Writes a contiguous subset of the local row to all remote nodes.

Definition at line 343 of file sst.hpp.

◆ put() [4/8]

template<class DerivedSST>
template<typename T >
void sst::SST< DerivedSST >::put ( SSTField< T > &  field)
inline

Writes a specific local field to all remote nodes.

Definition at line 349 of file sst.hpp.

◆ put() [5/8]

template<class DerivedSST>
template<typename T >
void sst::SST< DerivedSST >::put ( SSTFieldVector< T > &  vec_field)
inline

Writes a specific local vector field to all remote nodes.

Definition at line 355 of file sst.hpp.

◆ put() [6/8]

template<class DerivedSST>
template<typename T >
void sst::SST< DerivedSST >::put ( SSTFieldVector< T > &  vec_field,
std::size_t  index 
)
inline

Writes only a single element of a vector field to all remote nodes.

Definition at line 362 of file sst.hpp.

◆ put() [7/8]

template<class DerivedSST>
template<typename T >
void sst::SST< DerivedSST >::put ( const std::vector< uint32_t >  receiver_ranks,
SSTFieldVector< T > &  vec_field,
std::size_t  index 
)
inline

Writes only a single element of a vector field to only some of the remote nodes.

Definition at line 371 of file sst.hpp.

◆ put() [8/8]

template<typename DerivedSST >
void sst::SST< DerivedSST >::put ( const std::vector< uint32_t >  receiver_ranks,
size_t  offset,
size_t  size 
)

Writes a contiguous subset of the local row to some of the remote nodes.

Definition at line 144 of file sst_impl.hpp.

◆ put_with_completion() [1/4]

template<class DerivedSST>
void sst::SST< DerivedSST >::put_with_completion ( )
inline

Definition at line 329 of file sst.hpp.

◆ put_with_completion() [2/4]

template<class DerivedSST>
void sst::SST< DerivedSST >::put_with_completion ( const std::vector< uint32_t >  receiver_ranks)
inline

Definition at line 338 of file sst.hpp.

◆ put_with_completion() [3/4]

template<class DerivedSST>
void sst::SST< DerivedSST >::put_with_completion ( size_t  offset,
size_t  size 
)
inline

Definition at line 379 of file sst.hpp.

◆ put_with_completion() [4/4]

template<typename DerivedSST >
void sst::SST< DerivedSST >::put_with_completion ( const std::vector< uint32_t >  receiver_ranks,
size_t  offset,
size_t  size 
)

Definition at line 158 of file sst_impl.hpp.

◆ set_bases_and_rowLens() [1/2]

template<class DerivedSST>
void sst::SST< DerivedSST >::set_bases_and_rowLens ( char_p ,
const size_t   
)
inlineprivate

Definition at line 399 of file sst.hpp.

◆ set_bases_and_rowLens() [2/2]

template<class DerivedSST>
template<typename Field , typename... Fields>
void sst::SST< DerivedSST >::set_bases_and_rowLens ( char_p base,
const size_t  rlen,
Field &  f,
Fields &...  rest 
)
inlineprivate

Definition at line 402 of file sst.hpp.

◆ SSTInit()

template<class DerivedSST>
template<typename... Fields>
void sst::SST< DerivedSST >::SSTInit ( Fields &...  fields)
inline

Definition at line 268 of file sst.hpp.

◆ start_predicate_evaluation()

template<typename DerivedSST >
void sst::SST< DerivedSST >::start_predicate_evaluation ( )

Starts the predicate evaluation loop.

This simply unblocks the background thread that runs the predicate evaluation loop.

It must be called at some point after the the constructor in order for any registered predicates to trigger.

Definition at line 47 of file sst_impl.hpp.

◆ sync_with_members() [1/2]

template<typename DerivedSST >
void sst::SST< DerivedSST >::sync_with_members ( ) const

Does a TCP sync with each member of the SST.

Exchanges a single byte of data with each member of the SST group over the TCP (not RDMA) connection, in descending order of the members' node ranks.

This creates a synchronization barrier, since the TCP reads are blocking, and should be called after SST initialization to ensure all nodes have finished initializing their local SST code.

Definition at line 272 of file sst_impl.hpp.

◆ sync_with_members() [2/2]

template<typename DerivedSST >
void sst::SST< DerivedSST >::sync_with_members ( std::vector< uint32_t >  row_indices) const

Syncs with a subset of the members.

Same as before but syncs with only a subset of the members.

Definition at line 286 of file sst_impl.hpp.

Friends And Related Function Documentation

◆ Predicates< DerivedSST >

template<class DerivedSST>
friend class Predicates< DerivedSST >
friend

Definition at line 181 of file sst.hpp.

Member Data Documentation

◆ all_indices

template<class DerivedSST>
std::vector<uint32_t> sst::SST< DerivedSST >::all_indices
private

Definition at line 195 of file sst.hpp.

◆ background_threads

template<class DerivedSST>
std::vector<std::thread> sst::SST< DerivedSST >::background_threads
private

Definition at line 174 of file sst.hpp.

◆ derived_this

template<class DerivedSST>
DerivedSST* sst::SST< DerivedSST >::derived_this
private

Definition at line 172 of file sst.hpp.

◆ failure_upcall

template<class DerivedSST>
failure_upcall_t sst::SST< DerivedSST >::failure_upcall
private

The function to call when a remote node appears to have failed.

Definition at line 211 of file sst.hpp.

◆ freeze_mutex

template<class DerivedSST>
std::mutex sst::SST< DerivedSST >::freeze_mutex
private

Mutex for failure detection and row freezing.

Definition at line 213 of file sst.hpp.

◆ members

template<class DerivedSST>
const std::vector<uint32_t>& sst::SST< DerivedSST >::members
private

List of nodes in the SST; indexes are row numbers, values are node IDs.

Definition at line 192 of file sst.hpp.

◆ members_by_id

template<class DerivedSST>
std::map<uint32_t, int, std::greater<uint32_t> > sst::SST< DerivedSST >::members_by_id
private

Maps node IDs to SST row indexes.

Definition at line 199 of file sst.hpp.

◆ my_index

template<class DerivedSST>
unsigned int sst::SST< DerivedSST >::my_index
private

Index (row number) of this node in the SST.

Definition at line 197 of file sst.hpp.

◆ my_node_id

template<class DerivedSST>
uint32_t sst::SST< DerivedSST >::my_node_id
private

ID of this node in the system.

Definition at line 201 of file sst.hpp.

◆ num_frozen

template<class DerivedSST>
int sst::SST< DerivedSST >::num_frozen {0}
private

The number of rows that have been frozen.

Definition at line 209 of file sst.hpp.

◆ num_members

template<class DerivedSST>
const unsigned int sst::SST< DerivedSST >::num_members
private

Equal to members.size()

Definition at line 194 of file sst.hpp.

◆ poll_cq_timeout_ms

template<class DerivedSST>
const uint32_t sst::SST< DerivedSST >::poll_cq_timeout_ms
private

timeout settings for poll completion queue

Definition at line 185 of file sst.hpp.

◆ predicates

template<class DerivedSST>
Predicates<DerivedSST> sst::SST< DerivedSST >::predicates

Definition at line 180 of file sst.hpp.

◆ res_vec

template<class DerivedSST>
std::vector<std::unique_ptr<resources> > sst::SST< DerivedSST >::res_vec
private

RDMA resources vector, one for each member.

Definition at line 216 of file sst.hpp.

◆ row_is_frozen

template<class DerivedSST>
std::vector<bool> sst::SST< DerivedSST >::row_is_frozen
private

Map of queue pair number to row.

Useful for detecting failures. Array with one entry for each row index, tracking whether the row is marked frozen (meaning its corresponding remote node has crashed).

Definition at line 207 of file sst.hpp.

◆ rowLen

template<class DerivedSST>
size_t sst::SST< DerivedSST >::rowLen
private

Length of each row in this SST, in bytes.

Definition at line 190 of file sst.hpp.

◆ rows

template<class DerivedSST>
volatile char* sst::SST< DerivedSST >::rows
private

Pointer to memory where the SST rows are stored.

Definition at line 187 of file sst.hpp.

◆ thread_shutdown

template<class DerivedSST>
std::atomic<bool> sst::SST< DerivedSST >::thread_shutdown
private

Definition at line 175 of file sst.hpp.

◆ thread_start

template<class DerivedSST>
bool sst::SST< DerivedSST >::thread_start
private

Indicates whether the predicate evaluation thread should start after being forked in the constructor.

Definition at line 220 of file sst.hpp.

◆ thread_start_cv

template<class DerivedSST>
std::condition_variable sst::SST< DerivedSST >::thread_start_cv
private

Notified when the predicate evaluation thread should start.

Definition at line 224 of file sst.hpp.

◆ thread_start_mutex

template<class DerivedSST>
std::mutex sst::SST< DerivedSST >::thread_start_mutex
private

Mutex for thread_start_cv.

Definition at line 222 of file sst.hpp.


The documentation for this class was generated from the following files: