Derecho  0.9
Distributed systems toolkit for RDMA
Namespaces | Classes | Typedefs | Enumerations | Functions | Variables
derecho Namespace Reference

Namespaces

 gmssst
 
 rpc
 

Classes

class  _Group
 
struct  CallbackSet
 Bundles together a set of callback functions for message delivery events. More...
 
struct  Conf
 The single configuration file for derecho. More...
 
struct  CrossProductPolicy
 An alternate type of subgroup allocation policy for subgroup types whose membership will be defined as a cross-product of other subgroups. More...
 
class  DefaultSubgroupAllocator
 Functor of type shard_view_generator_t that implements the default subgroup allocation algorithm, parameterized based on a policy for each subgroup type (i.e. More...
 
struct  derecho_exception
 Base exception class for all exceptions raised by Derecho. More...
 
struct  DerechoParams
 Bundles together a set of low-level parameters for configuring Derecho groups. More...
 
class  DerechoSST
 The GMS and derecho_group will share the same SST for efficiency. More...
 
struct  empty_reference_exception
 Exception that means a reference-like type is "empty" (does not contain a valid object). More...
 
class  ExternalCaller
 
class  Group
 The top-level object for creating a Derecho group. More...
 
class  GroupProjection
 
class  GroupReference
 
struct  IDeserializationContext
 The Deserialization Interface to be implemented by user applications. More...
 
struct  invalid_node_exception
 Exception that means the user requested an operation targeting a specific node and that node was not as valid target, e.g. More...
 
struct  invalid_subgroup_exception
 Exception that means the user made an invalid request for a subgroup handle, such as by supplying an out-of-bounds subgroup index. More...
 
struct  JoinResponse
 Bundles together a JoinResponseCode and the leader's node ID, which it also needs to send to the new node that wants to join. More...
 
class  LockedQueue
 A little helper class that implements a threadsafe queue by requiring all clients to lock a mutex before accessing the queue. More...
 
class  LockedReference
 A little helper class that wraps together a reference and a lock on a related mutex. More...
 
struct  MessageBuffer
 Represents a block of memory used to store a message. More...
 
class  MulticastGroup
 Implements the low-level mechanics of tracking multicasts in a Derecho group, using RDMC to deliver messages and SST to track their arrival and stability. More...
 
class  PersistenceManager
 PersistenceManager is responsible for persisting all the data in a group. More...
 
class  PersistsFields
 This is a marker interface for user-defined Replicated Objects (i.e. More...
 
struct  RaggedTrim
 Represents the data needed to log a "ragged trim" decision to disk. More...
 
struct  RawObject
 An empty class to be used as the "replicated type" for a subgroup that doesn't implement a Replicated Object. More...
 
struct  RDMCMessage
 A structure containing an RDMC message (which consists of some bytes in a registered memory region) and some associated metadata. More...
 
class  Replicated
 
class  ReplicatedObject
 Common interface for all types of Replicated<T>, specifying some methods for state transfer and persistence. More...
 
class  RestartLeaderState
 
struct  RestartState
 
struct  ShardAllocationPolicy
 A data structure defining the parameters of the default subgroup allocation function for a single subgroup; it specifies how one subgroup is divided into shards. More...
 
class  ShardIterator
 
struct  SSTMessage
 
class  subgroup_provisioning_exception
 An exception that indicates that a subgroup membership function was unable to finish executing because its enclosing Group was not in a valid state. More...
 
struct  SubgroupAllocationPolicy
 A data structure defining the parameters of the default subgroup allocation function for a single subgroup type. More...
 
struct  SubgroupInfo
 Container for whatever information is needed to describe a Group's subgroups and shards. More...
 
struct  SubgroupSettings
 A collection of settings for a single subgroup that this node is a member of. More...
 
class  SubView
 The subset of a View associated with a single shard, or a single subgroup if the subgroup is non-sharded. More...
 
class  View
 
class  ViewManager
 

Typedefs

using subgroup_id_t = uint32_t
 Type alias for the internal Subgroup IDs generated by ViewManager. More...
 
using message_id_t = int32_t
 Type alias for a message's unique "sequence number" or index. More...
 
using subgroup_type_id_t = uint32_t
 Type of the numeric ID used to refer to subgroup types within a Group; this is currently computed as the index of the subgroup type within Group's template parameters. More...
 
using message_callback_t = std::function< void(subgroup_id_t, node_id_t, message_id_t, std::optional< std::pair< char *, long long int > >, persistent::version_t)>
 Alias for the type of std::function that is used for message delivery event callbacks. More...
 
using persistence_callback_t = std::function< void(subgroup_id_t, persistent::version_t)>
 
using rpc_handler_t = std::function< void(subgroup_id_t, node_id_t, char *, uint32_t)>
 
template<typename T >
using Factory = std::function< std::unique_ptr< T >(persistent::PersistentRegistry *)>
 The type of factory function the user must provide to the Group constructor, to construct each Replicated Object that is assigned to a subgroup. More...
 
using persistence_manager_make_version_func_t = std::function< void(const subgroup_id_t &, const persistent::version_t &, const HLC &)>
 
using persistence_manager_post_persist_func_t = std::function< void(const subgroup_id_t &, const persistent::version_t &)>
 
using persistence_manager_callbacks_t = std::tuple< persistence_manager_make_version_func_t, persistence_manager_post_persist_func_t >
 
using subgroup_post_next_version_func_t = std::function< void(const subgroup_id_t &, const persistent::version_t &, const uint64_t &)>
 
using persistence_request_t = std::tuple< subgroup_id_t, persistent::version_t >
 
using ragged_trim_map_t = std::map< subgroup_id_t, std::map< uint32_t, std::unique_ptr< RaggedTrim > >>
 List of logged ragged trim states, indexed by (subgroup ID, shard num), stored by pointer. More...
 
template<typename T >
using SharedLockedReference = LockedReference< std::shared_lock< std::shared_timed_mutex >, T >
 
using view_upcall_t = std::function< void(const View &)>
 
using vector_int64_2d = std::vector< std::vector< int64_t > >
 Type of a 2-dimensional vector used to store potential node IDs, or -1. More...
 
template<typename TargetType , typename... TypePack>
using contains = std::integral_constant< bool,(std::is_same< TargetType, TypePack >::value||...)>
 A type-trait-like template that provides a True member "value" if TargetType matches some type in TypePack (according to std::is_same), or provides a False member "value" if TargetType does not match anything in TypePack. More...
 
template<typename T >
using replicated_index_map = std::map< uint32_t, Replicated< T > >
 
template<typename T >
using has_persistent_fields = std::is_base_of< PersistsFields, T >
 A template whose member field "value" will be true if type T inherits from PersistsFields, and false otherwise. More...
 
using subgroup_shard_layout_t = std::vector< std::vector< SubView > >
 The data structure used to store a subgroups-and-shards layout for a single subgroup type (i.e. More...
 
using subgroup_allocation_map_t = std::map< std::type_index, subgroup_shard_layout_t >
 The data structure used to store the subgroups-and-shards layouts for all subgroup types in a Group (at least during the subgroup allocation process). More...
 
using shard_view_generator_t = std::function< subgroup_allocation_map_t(const std::vector< std::type_index > &, const std::unique_ptr< View > &, View &)>
 The type of a lambda function that generates subgroup and shard views for a Derecho group. More...
 
using lock_guard_t = std::lock_guard< std::mutex >
 
using unique_lock_t = std::unique_lock< std::mutex >
 
using shared_lock_t = std::shared_lock< std::shared_timed_mutex >
 

Enumerations

enum  Mode { Mode::ORDERED, Mode::UNORDERED }
 
enum  CommitMessage { CommitMessage::PREPARE, CommitMessage::COMMIT, CommitMessage::ABORT, CommitMessage::ACK }
 A type-safe set of messages that can be sent during two-phase commit. More...
 
enum  JoinResponseCode { JoinResponseCode::OK, JoinResponseCode::TOTAL_RESTART, JoinResponseCode::ID_IN_USE, JoinResponseCode::LEADER_REDIRECT }
 A set of status codes the group leader can respond with upon initially receiving a connection request from a new node. More...
 
enum  PORT_TYPE { GMS = 1, RPC, SST, RDMC }
 

Functions

const std::string & getConfString (const std::string &key)
 
const int16_t getConfInt16 (const std::string &key)
 
const uint16_t getConfUInt16 (const std::string &key)
 
const int32_t getConfInt32 (const std::string &key)
 
const uint32_t getConfUInt32 (const std::string &key)
 
const int64_t getConfInt64 (const std::string &key)
 
const uint64_t getConfUInt64 (const std::string &key)
 
const float getConfFloat (const std::string &key)
 
const double getConfDouble (const std::string &key)
 
const bool getConfBoolean (const std::string &key)
 
const bool hasCustomizedConfKey (const std::string &key)
 
template<typename T >
std::set< T > functional_insert (std::set< T > &a, const std::set< T > &b)
 Inserts set b into set a and returns the modified a. More...
 
template<typename T >
std::vector< T > functional_append (const std::vector< T > &original, const T &item)
 Base case for functional_append, with one argument. More...
 
template<typename T , typename... RestArgs>
std::vector< T > functional_append (const std::vector< T > &original, const T &first_item, RestArgs... rest_items)
 Returns a new std::vector value that is equal to the parameter std::vector with the rest of the arguments appended. More...
 
template<typename K1 , typename K2 , typename V >
std::size_t multimap_size (const std::map< K1, std::map< K2, V >> &multimap)
 Returns the size of a std::map of std::maps, by counting up the sizes of all the inner maps. More...
 
template<typename K , typename V >
std::list< K > keys_as_list (const std::map< K, V > &map)
 Constructs a std::list of the keys in a std::map, in the same order as they appear in the std::map. More...
 
template<typename Container >
std::size_t index_of (const Container &container, const typename Container::value_type &elem)
 Finds a value in a STL container, and returns the index of that value in the container. More...
 
template<typename MapType >
void kind_map_builder (MapType &)
 
template<typename MapType , typename FirstType , typename... RestTypes>
void kind_map_builder (MapType &map, Factory< FirstType > curr_factory, Factory< RestTypes >... rest_factories)
 Actual implementation of make_kind_map; needs to be a separate function because the only way to build a KindMap is with a void mutator function. More...
 
template<typename... Types>
mutils::KindMap< Factory, Types... > make_kind_map (Factory< Types >... factories)
 Constructs a KindMap<Factory, Types...> from a list of factories of those types. More...
 
struct __attribute__ ((__packed__)) header
 The header for an individual multicast message, which will always be the first sizeof(header) bytes in the message's data buffer. More...
 
std::string ragged_trim_filename (subgroup_id_t subgroup_num, uint32_t shard_num)
 Builds a filename to use for a RaggedTrim logged to disk using its subgroup and shard IDs. More...
 
uint64_t version_hashcode ()
 The function that computes the Derecho version hashcode. More...
 
template<uint32_t counter, typename TargetType , typename FirstType , typename... RestTypes>
constexpr uint32_t index_of_type_impl ()
 The function that implements index_of_type, which is separate to hide the "counter" template parameter (an implementation detail only used to maintain state across recursive calls). More...
 
std::unique_ptr< RawObjectraw_object_factory (persistent::PersistentRegistry *)
 An implementation of Factory<T> for RawObject, which is trivial because RawObjects have no state. More...
 
template<typename T >
std::unique_ptr< T > deep_pointer_copy (const std::unique_ptr< T > &to_copy)
 A copy constructor for objects owned by unique_ptr. More...
 
subgroup_allocation_map_t one_subgroup_entire_view (const std::vector< std::type_index > &subgroup_type_order, const std::unique_ptr< View > &prev_view, View &curr_view)
 A simple implementation of shard_view_generator_t that creates a single, un-sharded subgroup containing all the members of curr_view for every subgroup type in the list. More...
 
subgroup_allocation_map_t one_subgroup_entire_view_raw (const std::vector< std::type_index > &subgroup_type_order, const std::unique_ptr< View > &prev_view, View &curr_view)
 A simple implementation of shard_view_generator_t that returns a single, un-sharded subgroup in Unordered (Raw) mode containing all the members of curr_view for every type in the list. More...
 
ShardAllocationPolicy flexible_even_shards (int num_shards, int min_nodes_per_shard, int max_nodes_per_shard, const std::string &profile="default")
 Returns a ShardAllocationPolicy that specifies num_shards "flexible" or fault-tolerant shards, each of which has the same minimum number of nodes and maximum number of nodes. More...
 
ShardAllocationPolicy flexible_even_shards (const std::string &profile)
 Returns a ShardAllocationPolicy that specifies a set number of "flexible" shards, each of which has the same minimum and maximum number of nodes, using the values of the constants num_shards, min_nodes, and max_nodes in the section of the configuration file identified by the profile string. More...
 
ShardAllocationPolicy fixed_even_shards (int num_shards, int nodes_per_shard, const std::string &profile="default")
 Returns a ShardAllocationPolicy that specifies num_shards shards with the same fixed number of nodes in each shard; each shard must have exactly nodes_per_shard members. More...
 
ShardAllocationPolicy raw_fixed_even_shards (int num_shards, int nodes_per_shard, const std::string &profile="default")
 Returns a ShardAllocationPolicy that specifies num_shards shards with the same fixed number of nodes in each shard, and every shard running in "raw" delivery mode. More...
 
ShardAllocationPolicy custom_shards_policy (const std::vector< int > &min_nodes_by_shard, const std::vector< int > &max_nodes_by_shard, const std::vector< Mode > &delivery_modes_by_shard, const std::vector< std::string > &profiles_by_shard)
 Returns a ShardAllocationPolicy for a subgroup that has a different number of members in each shard, and possibly has each shard in a different delivery mode. More...
 
ShardAllocationPolicy custom_shard_policy (const std::vector< Mode > &delivery_modes_by_shard, const std::vector< std::string > &profiles_by_shard)
 Returns a ShardAllocationPolicy for a subgroup that has a different number of members in each shard, and possibly has each shard in a different delivery mode. More...
 
SubgroupAllocationPolicy one_subgroup_policy (const ShardAllocationPolicy &policy)
 Returns a SubgroupAllocationPolicy for a replicated type that only has a single subgroup. More...
 
SubgroupAllocationPolicy identical_subgroups_policy (int num_subgroups, const ShardAllocationPolicy &subgroup_policy)
 Returns a SubgroupAllocationPolicy for a replicated type that needs n subgroups with identical sharding policies. More...
 
template<class T , class U >
size_t index_of (T container, U elem)
 Helper function to find the index of an element in a container. More...
 

Variables

const int MAJOR_VERSION = DERECHO_MAJOR_VERSION
 The current major version number of the Derecho library, as defined by Git. More...
 
const int MINOR_VERSION = DERECHO_MINOR_VERSION
 The current minor version number of the Derecho library, as defined by Git. More...
 
const int PATCH_VERSION = DERECHO_PATCH_VERSION
 The current "patch" (more-minor) version number of the Derecho library, as defined by Git. More...
 
const int COMMITS_AHEAD_OF_VERSION = DERECHO_COMMITS_AHEAD
 If the currently-compiled version of the Derecho library is more recent than the last "release" version, this is the number of Git commits by which it is ahead. More...
 
const char * VERSION_STRING
 A constant C-style string containing the current Derecho library version in dot-separated format, e.g. More...
 
const char * VERSION_STRING_PLUS_COMMITS
 A constant C-style string containing the current Derecho library version in dot-separated format, plus the number of Git commits the code is ahead of the last "release" version, separated by a plus sign. More...
 
uint64_t my_version_hashcode = version_hashcode()
 A runtime constant (computed once during static initialization) that represents the current running version of the Derecho library as a platform- dependent hash code. More...
 
template<typename TargetType , typename... TypePack>
constexpr uint32_t index_of_type = index_of_type_impl<0, TargetType, TypePack...>()
 A compile-time "function" that computes the index of a type within a template parameter pack of types. More...
 
constexpr char num_shards_profile_field [] = "num_shards"
 
constexpr char min_nodes_profile_field [] = "min_nodes"
 
constexpr char max_nodes_profile_field [] = "max_nodes"
 

Class Documentation

◆ derecho::CallbackSet

struct derecho::CallbackSet

Bundles together a set of callback functions for message delivery events.

These will be invoked by MulticastGroup or ViewManager to hand control back to the client if it wants to implement custom logic to respond to each message's arrival. (Note, this is a client-facing constructor argument, not an internal data structure).

Definition at line 39 of file multicast_group.hpp.

Class Members
persistence_callback_t global_persistence_callback
message_callback_t global_stability_callback
persistence_callback_t local_persistence_callback

◆ derecho::CrossProductPolicy

struct derecho::CrossProductPolicy

An alternate type of subgroup allocation policy for subgroup types whose membership will be defined as a cross-product of other subgroups.

Each node in the source subgroup will be placed in T subgroups, one for each shard in the target subgroup (the target subgroup has T shards). Thus, if there are S members in the source subgroup, and T shards in the target subgroup, S * T subgroups of a single type will be created. The nodes in the source subgroup will be marked as the only senders in these subgroups. A node that has rank i within the source subgroup can send a multicast to shard j of the target subgroup by selecting the cross-product subgroup at index (i * T + j).

Definition at line 124 of file subgroup_functions.hpp.

Class Members
pair< type_index, uint32_t > source_subgroup The (type, index) pair identifying the "source" subgroup of the cross-product.

Each member of this subgroup will be a sender in T subgroups, where T is the number of shards in the target subgroup.

pair< type_index, uint32_t > target_subgroup The (type, index) pair identifying the "target" subgroup of the cross-product.

Each shard in this subgroup will have all of its members assigned to S subgroups as receivers, where S is the number of members in the source subgroup.

◆ derecho::JoinResponse

struct derecho::JoinResponse

Bundles together a JoinResponseCode and the leader's node ID, which it also needs to send to the new node that wants to join.

Definition at line 80 of file view_manager.hpp.

Class Members
JoinResponseCode code
node_id_t leader_id

◆ derecho::RDMCMessage

struct derecho::RDMCMessage

A structure containing an RDMC message (which consists of some bytes in a registered memory region) and some associated metadata.

Note that the metadata (sender_id, index, etc.) is only stored locally, not sent over the network with the message.

Definition at line 195 of file multicast_group.hpp.

Collaboration diagram for derecho::RDMCMessage:
Class Members
message_id_t index The message's index (relative to other messages sent by that sender).
MessageBuffer message_buffer The MessageBuffer that contains the message's body.
uint32_t sender_id The unique node ID of the message's sender.
long long unsigned int size The message's size in bytes.

◆ derecho::ShardAllocationPolicy

struct derecho::ShardAllocationPolicy

A data structure defining the parameters of the default subgroup allocation function for a single subgroup; it specifies how one subgroup is divided into shards.

Definition at line 65 of file subgroup_functions.hpp.

Collaboration diagram for derecho::ShardAllocationPolicy:
Class Members
bool even_shards Whether all shards should contain the same number of members.
int max_nodes_per_shard If even_shards is true, this is the maximum number of nodes per shard.
vector< int > max_num_nodes_by_shard If even_shards is false, this will contain an entry for each shard indicating the maximum number of members it should have.
int min_nodes_per_shard If even_shards is true, this is the minimum number of nodes per shard.

(Ignored if even_shards is false).

vector< int > min_num_nodes_by_shard If even_shards is false, this will contain an entry for each shard indicating the minimum number of members it should have.

(Ignored if even_shards is true).

vector< Mode > modes_by_shard If even_shards is false, this will contain an entry for each shard indicating which delivery mode it should use.

(Ignored if even_shards is true).

int num_shards The number of shards; set to 1 for a non-sharded subgroup.
vector< string > profiles_by_shard If even_shards is false, this will contain an entry for each shard indicating which profile it should use.

(Ignored if even_shards is true).

Mode shards_mode If even_shards is true, this is the delivery mode that will be used for every shard.

(Ignored if even_shards is false).

string shards_profile If even_shards is true, this is the profile that will be used for every shard.

(Ignore if even_shards is false).

◆ derecho::SSTMessage

struct derecho::SSTMessage

Definition at line 207 of file multicast_group.hpp.

Class Members
volatile char * buf Pointer to the message.
int32_t index The message's index (relative to other messages sent by that sender).
uint32_t sender_id The unique node ID of the message's sender.
long long unsigned int size The message's size in bytes.

◆ derecho::SubgroupAllocationPolicy

struct derecho::SubgroupAllocationPolicy

A data structure defining the parameters of the default subgroup allocation function for a single subgroup type.

Definition at line 102 of file subgroup_functions.hpp.

Collaboration diagram for derecho::SubgroupAllocationPolicy:
Class Members
bool identical_subgroups Whether all subgroups of this type will have an identical shard layout.
int num_subgroups The number of subgroups of the same Replicated type to create.
vector< ShardAllocationPolicy > shard_policy_by_subgroup If identical_subgroups is true, contains a single entry with the allocation policy for all subgroups of this type.

If identical_subgroups is false, contains an entry for each subgroup describing that subgroup's shards.

◆ derecho::SubgroupSettings

struct derecho::SubgroupSettings

A collection of settings for a single subgroup that this node is a member of.

Mostly extracted from SubView, but tailored specifically to what MulticastGroup needs to know about subgroups and shards.

Definition at line 223 of file multicast_group.hpp.

Collaboration diagram for derecho::SubgroupSettings:
Class Members
vector< node_id_t > members The members of the subgroup.
Mode mode The operation mode of the subgroup.
uint32_t num_received_offset The offset of this node's num_received counter within the subgroup's SST section.
DerechoParams profile
int sender_rank This node's sender rank within the subgroup (as defined by SubView::sender_rank_of)
vector< int > senders The "is_sender" flags for members of the subgroup.
uint32_t shard_num This node's shard number within the subgroup.
uint32_t shard_rank This node's rank within its shard of the subgroup.
uint32_t slot_offset The offset of this node's slot within the subgroup's SST section.

Typedef Documentation

◆ contains

template<typename TargetType , typename... TypePack>
using derecho::contains = typedef std::integral_constant<bool, (std::is_same<TargetType, TypePack>::value || ...)>

A type-trait-like template that provides a True member "value" if TargetType matches some type in TypePack (according to std::is_same), or provides a False member "value" if TargetType does not match anything in TypePack.

Definition at line 63 of file group.hpp.

◆ Factory

template<typename T >
using derecho::Factory = typedef std::function<std::unique_ptr<T>(persistent::PersistentRegistry*)>

The type of factory function the user must provide to the Group constructor, to construct each Replicated Object that is assigned to a subgroup.

Definition at line 48 of file derecho_internal.hpp.

◆ has_persistent_fields

template<typename T >
using derecho::has_persistent_fields = typedef std::is_base_of<PersistsFields, T>

A template whose member field "value" will be true if type T inherits from PersistsFields, and false otherwise.

Just a convenient specialization of std::is_base_of.

Definition at line 43 of file replicated.hpp.

◆ lock_guard_t

using derecho::lock_guard_t = typedef std::lock_guard<std::mutex>

Definition at line 24 of file view_manager.cpp.

◆ message_callback_t

using derecho::message_callback_t = typedef std::function<void(subgroup_id_t, node_id_t, message_id_t, std::optional<std::pair<char*, long long int> >, persistent::version_t)>

Alias for the type of std::function that is used for message delivery event callbacks.

Definition at line 41 of file derecho_internal.hpp.

◆ message_id_t

using derecho::message_id_t = typedef int32_t

Type alias for a message's unique "sequence number" or index.

This allows us to change exactly which numeric type we use to store it.

Definition at line 31 of file derecho_internal.hpp.

◆ persistence_callback_t

Definition at line 42 of file derecho_internal.hpp.

◆ persistence_manager_callbacks_t

Definition at line 59 of file derecho_internal.hpp.

◆ persistence_manager_make_version_func_t

using derecho::persistence_manager_make_version_func_t = typedef std::function<void( const subgroup_id_t&, const persistent::version_t&, const HLC&)>

Definition at line 54 of file derecho_internal.hpp.

◆ persistence_manager_post_persist_func_t

using derecho::persistence_manager_post_persist_func_t = typedef std::function<void( const subgroup_id_t&, const persistent::version_t&)>

Definition at line 57 of file derecho_internal.hpp.

◆ persistence_request_t

Definition at line 23 of file persistence_manager.hpp.

◆ ragged_trim_map_t

using derecho::ragged_trim_map_t = typedef std::map<subgroup_id_t, std::map<uint32_t, std::unique_ptr<RaggedTrim> >>

List of logged ragged trim states, indexed by (subgroup ID, shard num), stored by pointer.

Definition at line 55 of file restart_state.hpp.

◆ replicated_index_map

template<typename T >
using derecho::replicated_index_map = typedef std::map<uint32_t, Replicated<T> >

Definition at line 67 of file group.hpp.

◆ rpc_handler_t

using derecho::rpc_handler_t = typedef std::function<void(subgroup_id_t, node_id_t, char*, uint32_t)>

Definition at line 43 of file derecho_internal.hpp.

◆ shard_view_generator_t

using derecho::shard_view_generator_t = typedef std::function<subgroup_allocation_map_t( const std::vector<std::type_index>&, const std::unique_ptr<View>&, View&)>

The type of a lambda function that generates subgroup and shard views for a Derecho group.

Parameters
conststd::vector<std::type_index>& - A list of type_indexes of all the subgroup types in this Group, arranged in the same order as the template parameters to Group. This is the order in which the subgroup types should be allocated.
conststd::unique_ptr<View>& - A pointer to the previous View, if there is one
View&- A reference to the current View, which is in the process of being provisioned into subgroups and shards
Returns
subgroup_allocation_map_t - A subgroup allocation map in which each subgroup type in the Group has an entry containing its subgroup and shard SubViews. Thus, there should be a key in this map for every element in the std::vector<std::type_index> that is the first parameter.
Exceptions
subgroup_provisioning_exceptionif there are not enough nodes in the current view to allocate all the subgroups and shards

Definition at line 83 of file subgroup_info.hpp.

◆ shared_lock_t

using derecho::shared_lock_t = typedef std::shared_lock<std::shared_timed_mutex>

Definition at line 26 of file view_manager.cpp.

◆ SharedLockedReference

template<typename T >
using derecho::SharedLockedReference = typedef LockedReference<std::shared_lock<std::shared_timed_mutex>, T>

Definition at line 86 of file view_manager.hpp.

◆ subgroup_allocation_map_t

using derecho::subgroup_allocation_map_t = typedef std::map<std::type_index, subgroup_shard_layout_t>

The data structure used to store the subgroups-and-shards layouts for all subgroup types in a Group (at least during the subgroup allocation process).

The keys are Replicated Object types, and the values are subgroups-and-shards layouts as defined in subgroup_shard_layout_t.

Note that since the subgroup_shard_layout_t elements are stored by value, placing a local variable of type subgroup_shard_layout_t in the map will copy all of the vectors and SubViews; it is preferable to use std::move to move-assign the local variable into the map, like this: subgroup_allocation.emplace(subgroup_type, std::move(subgroup_layout));

Combining this type with subgroup_shard_layout_t, the map is organized by: subgroup type -> subgroup index -> shard index -> sub-view of that shard

Definition at line 60 of file subgroup_info.hpp.

◆ subgroup_id_t

using derecho::subgroup_id_t = typedef uint32_t

Type alias for the internal Subgroup IDs generated by ViewManager.

This allows us to change exactly which numeric type we use to store it.

Definition at line 28 of file derecho_internal.hpp.

◆ subgroup_post_next_version_func_t

using derecho::subgroup_post_next_version_func_t = typedef std::function<void( const subgroup_id_t&, const persistent::version_t&, const uint64_t&)>

Definition at line 65 of file derecho_internal.hpp.

◆ subgroup_shard_layout_t

using derecho::subgroup_shard_layout_t = typedef std::vector<std::vector<SubView> >

The data structure used to store a subgroups-and-shards layout for a single subgroup type (i.e.

Replicated Object type). The outer vector represents subgroups of the same type, and the inner vector represents shards of each subgroup, so the vectors map subgroup index -> shard index -> sub-view of that shard.

Definition at line 43 of file subgroup_info.hpp.

◆ subgroup_type_id_t

using derecho::subgroup_type_id_t = typedef uint32_t

Type of the numeric ID used to refer to subgroup types within a Group; this is currently computed as the index of the subgroup type within Group's template parameters.

Definition at line 37 of file derecho_internal.hpp.

◆ unique_lock_t

using derecho::unique_lock_t = typedef std::unique_lock<std::mutex>

Definition at line 25 of file view_manager.cpp.

◆ vector_int64_2d

using derecho::vector_int64_2d = typedef std::vector<std::vector<int64_t> >

Type of a 2-dimensional vector used to store potential node IDs, or -1.

Definition at line 91 of file view_manager.hpp.

◆ view_upcall_t

using derecho::view_upcall_t = typedef std::function<void(const View&)>

Definition at line 88 of file view_manager.hpp.

Enumeration Type Documentation

◆ CommitMessage

A type-safe set of messages that can be sent during two-phase commit.

Enumerator
PREPARE 

PREPARE.

COMMIT 

COMMIT.

ABORT 

ABORT.

ACK 

ACK.

Definition at line 38 of file restart_state.hpp.

◆ JoinResponseCode

A set of status codes the group leader can respond with upon initially receiving a connection request from a new node.

Enumerator
OK 

OK The new member can proceed to join as normal.

TOTAL_RESTART 

TOTAL_RESTART The group is currently restarting from a total failure, so the new member should send its logged view and ragged trim.

ID_IN_USE 

ID_IN_USE The node's ID is already listed as a member of the current view, so it can't join.

LEADER_REDIRECT 

LEADER_REDIRECT This node is not actually the leader and can't accept a join.

Definition at line 69 of file view_manager.hpp.

◆ Mode

enum derecho::Mode
strong
Enumerator
ORDERED 
UNORDERED 

Definition at line 4 of file derecho_modes.hpp.

◆ PORT_TYPE

Enumerator
GMS 
RPC 
SST 
RDMC 

Definition at line 22 of file view.hpp.

Function Documentation

◆ __attribute__()

struct derecho::__attribute__ ( (__packed__)  )

The header for an individual multicast message, which will always be the first sizeof(header) bytes in the message's data buffer.

Definition at line 49 of file multicast_group.hpp.

◆ custom_shard_policy()

ShardAllocationPolicy derecho::custom_shard_policy ( const std::vector< Mode > &  delivery_modes_by_shard,
const std::vector< std::string > &  profiles_by_shard 
)

Returns a ShardAllocationPolicy for a subgroup that has a different number of members in each shard, and possibly has each shard in a different delivery mode.

In this function, the minimum and maximum number of members in each shard is specified by the min_nodes and max_nodes constants in each shard's configuration profile.

Parameters
delivery_modes_by_shardA vector specifying the delivery mode (Raw or Ordered) for each shard
profiles_by_shardA vector specifying the configuration profile to use for each shard
Returns
A ShardAllocationPolicy that specifies the shard sizes and modes found in the configuration profiles

Definition at line 67 of file subgroup_functions.cpp.

◆ custom_shards_policy()

ShardAllocationPolicy derecho::custom_shards_policy ( const std::vector< int > &  min_nodes_by_shard,
const std::vector< int > &  max_nodes_by_shard,
const std::vector< Mode > &  delivery_modes_by_shard,
const std::vector< std::string > &  profiles_by_shard 
)

Returns a ShardAllocationPolicy for a subgroup that has a different number of members in each shard, and possibly has each shard in a different delivery mode.

Note that the parameter vectors must all be the same length.

Parameters
min_nodes_by_shardA vector specifying the minimum number of nodes for each shard; the ith shard must have at least min_nodes_by_shard[i] members.
max_nodes_by_shardA vector specifying the maximum number of nodes for each shard; the ith shard can have up to max_nodes_by_shard[i] members.
delivery_modes_by_shardA vector specifying the delivery mode (Raw or Ordered) for each shard, in the same order as the other vectors.
profiles_by_shardA vector specifying the profile (defined in configuration file) for each shard, in the same order as the other vectors.
Returns
A ShardAllocationPolicy that specifies these shard sizes and modes.

Definition at line 80 of file subgroup_functions.cpp.

◆ deep_pointer_copy()

template<typename T >
std::unique_ptr<T> derecho::deep_pointer_copy ( const std::unique_ptr< T > &  to_copy)

A copy constructor for objects owned by unique_ptr.

Does the obvious thing and invokes the copy constructor of the object being pointed to, or returns nullptr if the unique_ptr is empty.

Parameters
to_copyA unique_ptr to the object to copy
Returns
A new object in a new unique_ptr that is a copy of the old object.

Definition at line 26 of file subgroup_functions.hpp.

◆ fixed_even_shards()

ShardAllocationPolicy derecho::fixed_even_shards ( int  num_shards,
int  nodes_per_shard,
const std::string &  profile = "default" 
)

Returns a ShardAllocationPolicy that specifies num_shards shards with the same fixed number of nodes in each shard; each shard must have exactly nodes_per_shard members.

Parameters
num_shardsThe number of shards to request in this policy.
nodes_per_shardThe number of nodes per shard to request.
profileAn optional configuration profile string to associate with every shard in this subgroup
Returns
A ShardAllocationPolicy value with these parameters.

Definition at line 55 of file subgroup_functions.cpp.

◆ flexible_even_shards() [1/2]

ShardAllocationPolicy derecho::flexible_even_shards ( int  num_shards,
int  min_nodes_per_shard,
int  max_nodes_per_shard,
const std::string &  profile = "default" 
)

Returns a ShardAllocationPolicy that specifies num_shards "flexible" or fault-tolerant shards, each of which has the same minimum number of nodes and maximum number of nodes.

Parameters
num_shardsThe number of shards to request in this policy
min_nodes_per_shardThe minimum number of nodes that each shard can have
max_nodes_per_shardThe maximum number of nodes that each shard can have
profileAn optional configuration profile string to associate with every shard in this subgroup
Returns
A ShardAllocationPolicy value with these parameters

Definition at line 48 of file subgroup_functions.cpp.

◆ flexible_even_shards() [2/2]

ShardAllocationPolicy derecho::flexible_even_shards ( const std::string &  profile)

Returns a ShardAllocationPolicy that specifies a set number of "flexible" shards, each of which has the same minimum and maximum number of nodes, using the values of the constants num_shards, min_nodes, and max_nodes in the section of the configuration file identified by the profile string.

Parameters
profileA profile string identifying the configuration of this subgroup
Returns
A ShardAllocationPolicy value constructed based on this configuration profile

Definition at line 40 of file subgroup_functions.cpp.

◆ functional_append() [1/2]

template<typename T >
std::vector<T> derecho::functional_append ( const std::vector< T > &  original,
const T &  item 
)

Base case for functional_append, with one argument.

Definition at line 34 of file container_template_functions.hpp.

◆ functional_append() [2/2]

template<typename T , typename... RestArgs>
std::vector<T> derecho::functional_append ( const std::vector< T > &  original,
const T &  first_item,
RestArgs...  rest_items 
)

Returns a new std::vector value that is equal to the parameter std::vector with the rest of the arguments appended.

Adds some missing functionality to std::vector: the ability to append to a const vector without taking a several-line detour to call the void emplace_back() method.

Parameters
originalThe vector that should be the prefix of the new vector
first_itemThe first element to append to the original vector
rest_itemsThe rest of the elements to append to the original vector
Returns
A new vector (by value), containing a copy of original plus all the elements given as arguments.

Definition at line 52 of file container_template_functions.hpp.

◆ functional_insert()

template<typename T >
std::set<T> derecho::functional_insert ( std::set< T > &  a,
const std::set< T > &  b 
)

Inserts set b into set a and returns the modified a.

Hack to get around the fact that set::insert doesn't return *this.

Parameters
aThe set to modify and return
bThe set that should be inserted into a.
Returns
Set a.

Definition at line 25 of file container_template_functions.hpp.

◆ getConfBoolean()

const bool derecho::getConfBoolean ( const std::string &  key)

Definition at line 146 of file conf.cpp.

◆ getConfDouble()

const double derecho::getConfDouble ( const std::string &  key)

Definition at line 142 of file conf.cpp.

◆ getConfFloat()

const float derecho::getConfFloat ( const std::string &  key)

Definition at line 138 of file conf.cpp.

◆ getConfInt16()

const int16_t derecho::getConfInt16 ( const std::string &  key)

Definition at line 122 of file conf.cpp.

◆ getConfInt32()

const int32_t derecho::getConfInt32 ( const std::string &  key)

Definition at line 114 of file conf.cpp.

◆ getConfInt64()

const int64_t derecho::getConfInt64 ( const std::string &  key)

Definition at line 130 of file conf.cpp.

◆ getConfString()

const std::string & derecho::getConfString ( const std::string &  key)

Definition at line 110 of file conf.cpp.

◆ getConfUInt16()

const uint16_t derecho::getConfUInt16 ( const std::string &  key)

Definition at line 126 of file conf.cpp.

◆ getConfUInt32()

const uint32_t derecho::getConfUInt32 ( const std::string &  key)

Definition at line 118 of file conf.cpp.

◆ getConfUInt64()

const uint64_t derecho::getConfUInt64 ( const std::string &  key)

Definition at line 134 of file conf.cpp.

◆ hasCustomizedConfKey()

const bool derecho::hasCustomizedConfKey ( const std::string &  key)

Definition at line 150 of file conf.cpp.

◆ identical_subgroups_policy()

SubgroupAllocationPolicy derecho::identical_subgroups_policy ( int  num_subgroups,
const ShardAllocationPolicy subgroup_policy 
)

Returns a SubgroupAllocationPolicy for a replicated type that needs n subgroups with identical sharding policies.

Parameters
num_subgroupsThe number of subgroups to create.
subgroup_policyThe policy to use for sharding each subgroup.
Returns
A SubgroupAllocationPolicy for a replicated type with num_subgroups copies of the same subgroup.

Definition at line 93 of file subgroup_functions.cpp.

◆ index_of() [1/2]

template<class T , class U >
size_t derecho::index_of ( container,
elem 
)

Helper function to find the index of an element in a container.

Definition at line 19 of file multicast_group.cpp.

◆ index_of() [2/2]

template<typename Container >
std::size_t derecho::index_of ( const Container &  container,
const typename Container::value_type &  elem 
)

Finds a value in a STL container, and returns the index of that value in the container.

This simply combines std::find (which returns an opaque pointer to the found value) with std::distance (which converts the pointer to a numeric index).

Parameters
containerAn STL container, which must provide a member type value_type
elemAn element of type value_type to search for in the container
Returns
The index of the element within the container, or 1 greater than the size of the container if the element was not found.

Definition at line 99 of file container_template_functions.hpp.

◆ index_of_type_impl()

template<uint32_t counter, typename TargetType , typename FirstType , typename... RestTypes>
constexpr uint32_t derecho::index_of_type_impl ( )

The function that implements index_of_type, which is separate to hide the "counter" template parameter (an implementation detail only used to maintain state across recursive calls).

Definition at line 39 of file group.hpp.

◆ keys_as_list()

template<typename K , typename V >
std::list<K> derecho::keys_as_list ( const std::map< K, V > &  map)

Constructs a std::list of the keys in a std::map, in the same order as they appear in the std::map.

Parameters
mapA std::map
Returns
A std::list containing a copy of each key in the map, in the same order

Definition at line 80 of file container_template_functions.hpp.

◆ kind_map_builder() [1/2]

template<typename MapType >
void derecho::kind_map_builder ( MapType &  )

Definition at line 17 of file make_kind_map.hpp.

◆ kind_map_builder() [2/2]

template<typename MapType , typename FirstType , typename... RestTypes>
void derecho::kind_map_builder ( MapType &  map,
Factory< FirstType >  curr_factory,
Factory< RestTypes >...  rest_factories 
)

Actual implementation of make_kind_map; needs to be a separate function because the only way to build a KindMap is with a void mutator function.

Parameters
mapA mutable reference to the KindMap being constructed
curr_factoryThe first factory in the parameter pack of factories
rest_factoriesThe rest of the parameter pack

Definition at line 27 of file make_kind_map.hpp.

◆ make_kind_map()

template<typename... Types>
mutils::KindMap<Factory, Types...> derecho::make_kind_map ( Factory< Types >...  factories)

Constructs a KindMap<Factory, Types...> from a list of factories of those types.

Could probably be made even more generic, to construct a KindMap of any template given a list of objects that match that template, but that would involve writing a template template parameter, which is too much black magic for me to understand.

Parameters
factoriesOne instance of Factory<T> for each T in the type list
Returns
A KindMap of factories, mapping each type to a Factory for that type.

Definition at line 43 of file make_kind_map.hpp.

◆ multimap_size()

template<typename K1 , typename K2 , typename V >
std::size_t derecho::multimap_size ( const std::map< K1, std::map< K2, V >> &  multimap)

Returns the size of a std::map of std::maps, by counting up the sizes of all the inner maps.

Sort of a "deep size" for the common case of a map-of-maps.

Parameters
multimapA map of maps
Returns
The total number of elements in the std::map, counting all the elements in all the "inner" std::maps.

Definition at line 65 of file container_template_functions.hpp.

◆ one_subgroup_entire_view()

subgroup_allocation_map_t derecho::one_subgroup_entire_view ( const std::vector< std::type_index > &  subgroup_type_order,
const std::unique_ptr< View > &  prev_view,
View curr_view 
)

A simple implementation of shard_view_generator_t that creates a single, un-sharded subgroup containing all the members of curr_view for every subgroup type in the list.

This is best used when there is only one subgroup type.

Definition at line 17 of file subgroup_functions.cpp.

◆ one_subgroup_entire_view_raw()

subgroup_allocation_map_t derecho::one_subgroup_entire_view_raw ( const std::vector< std::type_index > &  subgroup_type_order,
const std::unique_ptr< View > &  prev_view,
View curr_view 
)

A simple implementation of shard_view_generator_t that returns a single, un-sharded subgroup in Unordered (Raw) mode containing all the members of curr_view for every type in the list.

This is best used when there is only one subgroup type.

Definition at line 29 of file subgroup_functions.cpp.

◆ one_subgroup_policy()

SubgroupAllocationPolicy derecho::one_subgroup_policy ( const ShardAllocationPolicy policy)

Returns a SubgroupAllocationPolicy for a replicated type that only has a single subgroup.

The ShardAllocationPolicy argument can be the result of one of the ShardAllocationPolicy helper functions.

Parameters
policyThe allocation policy to use for the single subgroup.
Returns
A SubgroupAllocationPolicy for a single-subgroup type.

Definition at line 89 of file subgroup_functions.cpp.

◆ ragged_trim_filename()

std::string derecho::ragged_trim_filename ( subgroup_id_t  subgroup_num,
uint32_t  shard_num 
)
inline

Builds a filename to use for a RaggedTrim logged to disk using its subgroup and shard IDs.

Definition at line 48 of file restart_state.hpp.

◆ raw_fixed_even_shards()

ShardAllocationPolicy derecho::raw_fixed_even_shards ( int  num_shards,
int  nodes_per_shard,
const std::string &  profile = "default" 
)

Returns a ShardAllocationPolicy that specifies num_shards shards with the same fixed number of nodes in each shard, and every shard running in "raw" delivery mode.

Parameters
num_shardsThe number of shards to request in this policy.
nodes_per_shardThe number of nodes per shard to request.
profileAn optional configuration profile string to associate with every shard in this subgroup
Returns
A ShardAllocationPolicy value with these parameters.

Definition at line 61 of file subgroup_functions.cpp.

◆ raw_object_factory()

std::unique_ptr<RawObject> derecho::raw_object_factory ( persistent::PersistentRegistry )
inline

An implementation of Factory<T> for RawObject, which is trivial because RawObjects have no state.

Definition at line 59 of file replicated.hpp.

◆ version_hashcode()

uint64_t derecho::version_hashcode ( )

The function that computes the Derecho version hashcode.

It should not be necessary to call this function at runtime, since its output will not change and the value is already available in the global variable my_version_hashcode.

Returns
The current Derecho version hashcode.

Definition at line 86 of file version_code.cpp.

Variable Documentation

◆ COMMITS_AHEAD_OF_VERSION

const int derecho::COMMITS_AHEAD_OF_VERSION = DERECHO_COMMITS_AHEAD

If the currently-compiled version of the Derecho library is more recent than the last "release" version, this is the number of Git commits by which it is ahead.

This is updated when the library is compiled.

Definition at line 35 of file git_version.cpp.

◆ index_of_type

template<typename TargetType , typename... TypePack>
constexpr uint32_t derecho::index_of_type = index_of_type_impl<0, TargetType, TypePack...>()
inline

A compile-time "function" that computes the index of a type within a template parameter pack of types.

The value of this constant is equal to the index of TargetType within TypePack. (The compiler will spew template errors if TargetType is not actually in TypePack).

Template Parameters
TargetTypeThe type to search for in the parameter pack
TypePackThe template parameter pack that should be searched

Definition at line 55 of file group.hpp.

◆ MAJOR_VERSION

const int derecho::MAJOR_VERSION = DERECHO_MAJOR_VERSION

The current major version number of the Derecho library, as defined by Git.

This is updated when the library is compiled.

Definition at line 32 of file git_version.cpp.

◆ max_nodes_profile_field

constexpr char derecho::max_nodes_profile_field[] = "max_nodes"

Definition at line 40 of file subgroup_functions.hpp.

◆ min_nodes_profile_field

constexpr char derecho::min_nodes_profile_field[] = "min_nodes"

Definition at line 39 of file subgroup_functions.hpp.

◆ MINOR_VERSION

const int derecho::MINOR_VERSION = DERECHO_MINOR_VERSION

The current minor version number of the Derecho library, as defined by Git.

This is updated when the library is compiled.

Definition at line 33 of file git_version.cpp.

◆ my_version_hashcode

uint64_t derecho::my_version_hashcode = version_hashcode()

A runtime constant (computed once during static initialization) that represents the current running version of the Derecho library as a platform- dependent hash code.

This can be used to check that two copies of a Derecho program are using the same version of the library and were compiled for the same architecture, since the hash codes will not match if they are computed on different architectures.

Definition at line 98 of file version_code.cpp.

◆ num_shards_profile_field

constexpr char derecho::num_shards_profile_field[] = "num_shards"

Definition at line 38 of file subgroup_functions.hpp.

◆ PATCH_VERSION

const int derecho::PATCH_VERSION = DERECHO_PATCH_VERSION

The current "patch" (more-minor) version number of the Derecho library, as defined by Git.

This is updated when the library is compiled.

Definition at line 34 of file git_version.cpp.

◆ VERSION_STRING

const char * derecho::VERSION_STRING
Initial value:
"."
"."
#define DERECHO_PATCH_VERSION
Definition: git_version.cpp:11
#define DERECHO_MAJOR_VERSION
Definition: git_version.cpp:15
#define QUOTE(x)
Definition: git_version.cpp:28

A constant C-style string containing the current Derecho library version in dot-separated format, e.g.

"1.0.0"

Definition at line 37 of file git_version.cpp.

◆ VERSION_STRING_PLUS_COMMITS

const char * derecho::VERSION_STRING_PLUS_COMMITS
Initial value:
"."
"."
"+"
#define DERECHO_MAJOR_VERSION
Definition: git_version.cpp:15
#define QUOTE(x)
Definition: git_version.cpp:28
#define DERECHO_COMMITS_AHEAD
Definition: git_version.cpp:12

A constant C-style string containing the current Derecho library version in dot-separated format, plus the number of Git commits the code is ahead of the last "release" version, separated by a plus sign.

E.g. "1.0.0+5"

Definition at line 42 of file git_version.cpp.