消除Learner::SendNowInstanceID不必要的拷贝

[dyx2025]

消除Learner::SendNowInstanceID不必要的拷贝

Learner::SendNowInstanceID分别通过SystemVSM::GetCheckpointBuffer和MasterStateMachine::GetCheckpointBuffer获取各自的checkpoint buffer。然后把各自的checkpoint buffer分别拷贝到oPaxosMsg的systemvariables和mastervariables字段。这个两个checkpoint buffer都是临时的字符串对象，可以把这两个临时字符串移动到systemvariables和mastervariables字段，从而避免不必要的拷贝。

为什么没有在SystemVSM::GetCheckpointBuffer和MasterStateMachine::GetCheckpointBuffer分别把protocol buffers对象直接序列化oPaxosMsg的systemvariables和mastervariables字段呢？源于以下一些考量。一来没有在protocol buffers官方文档找到关于SerializeToString失败时关于字符串入参的描述。二来Learner::SendNowInstanceID在SystemVSM::GetCheckpointBuffer和MasterStateMachine::GetCheckpointBuffer失败时继续走后面的发消息逻辑。如果SerializeToString失败了(即SystemVSM::GetCheckpointBuffer和MasterStateMachine::GetCheckpointBuffer失败)，而作为入参的systemvariables和mastervariables字段填入了一些脏数据，那么这些脏数据会随着后面的发送消息逻辑传播到其他地方，有可能会影响到phxpaxos的正常运行。

sSystemVariablesCPBuffer和sMasterVariablesCPBuffer因为SSO(小字符串优化)把序列化的结果存储在栈空间上，这导致了无法移动，这情况对性能影响的如何？无法移动则退化成拷贝，std::move逻辑相当于退化成原来的set_systemvariables和set_mastervariables。理论上，无法移动时的性能相当于原来的性能。

src/config/system_v_sm.cpp

int SystemVSM :: GetCheckpointBuffer(std::string & sCPBuffer)                                                                                                                                               
{
    if (m_oSystemVariables.version() == (uint64_t)-1
            || m_oSystemVariables.gid() == 0)
    {   
        return 0;
    }   
        
    bool sSucc = m_oSystemVariables.SerializeToString(&sCPBuffer);
    if (!sSucc)
    {   
        PLG1Err("Variables.Serialize fail");
        return -1; 
    }   

    return 0;
}

src/master/master_sm.cpp

int MasterStateMachine :: GetCheckpointBuffer(std::string & sCPBuffer)                                                                                                                                      
{
    std::lock_guard<std::mutex> oLockGuard(m_oMutex);

    if (m_llMasterVersion == (uint64_t)-1)
    {   
        return 0;
    }   
         
    MasterVariables oVariables;
    oVariables.set_masternodeid(m_iMasterNodeID);
    oVariables.set_version(m_llMasterVersion);
    oVariables.set_leasetime(m_iLeaseTime);
        
    bool sSucc = oVariables.SerializeToString(&sCPBuffer);
    if (!sSucc)
    {   
        PLG1Err("Variables.Serialize fail");
        return -1; 
    }   

    return 0;
}

src/algorithm/learner.cpp

void Learner :: SendNowInstanceID(const uint64_t llInstanceID, const nodeid_t iSendNodeID)
{
    BP->GetLearnerBP()->SendNowInstanceID();

    PaxosMsg oPaxosMsg;
    oPaxosMsg.set_instanceid(llInstanceID);
    oPaxosMsg.set_nodeid(m_poConfig->GetMyNodeID());
    oPaxosMsg.set_msgtype(MsgType_PaxosLearner_SendNowInstanceID);
    oPaxosMsg.set_nowinstanceid(GetInstanceID());
    oPaxosMsg.set_minchoseninstanceid(m_poCheckpointMgr->GetMinChosenInstanceID());

    if ((GetInstanceID() - llInstanceID) > 50) 
    {
        // Learner::SendNowInstanceID分别通过SystemVSM::GetCheckpointBuffer和MasterStateMachine::GetCheckpointBuffer获取各自的checkpoint buffer。然后把各自的checkpoint buffer分别拷贝到oPaxosMsg的systemvariables和mastervariables字段                                                                                                                                                                                                       
        //instanceid too close not need to send vsm/master checkpoint. 
        string sSystemVariablesCPBuffer;
        int ret = m_poConfig->GetSystemVSM()->GetCheckpointBuffer(sSystemVariablesCPBuffer);
        if (ret == 0)
        {   
            oPaxosMsg.set_systemvariables(sSystemVariablesCPBuffer);
        }   

        string sMasterVariablesCPBuffer;
        if (m_poConfig->GetMasterSM() != nullptr)
        {   
            int ret = m_poConfig->GetMasterSM()->GetCheckpointBuffer(sMasterVariablesCPBuffer);
            if (ret == 0)
            {   
                oPaxosMsg.set_mastervariables(sMasterVariablesCPBuffer);
            }   
        }   
    }   

    SendMessage(iSendNodeID, oPaxosMsg);
}

修改后的代码路径：
src/algorithm/learner.cpp

void Learner :: SendNowInstanceID(const uint64_t llInstanceID, const nodeid_t iSendNodeID)
{
    BP->GetLearnerBP()->SendNowInstanceID();

    PaxosMsg oPaxosMsg;
    oPaxosMsg.set_instanceid(llInstanceID);
    oPaxosMsg.set_nodeid(m_poConfig->GetMyNodeID());
    oPaxosMsg.set_msgtype(MsgType_PaxosLearner_SendNowInstanceID);
    oPaxosMsg.set_nowinstanceid(GetInstanceID());
    oPaxosMsg.set_minchoseninstanceid(m_poCheckpointMgr->GetMinChosenInstanceID());

    if ((GetInstanceID() - llInstanceID) > 50) 
    {                                                                                                                                                                                                       
        //instanceid too close not need to send vsm/master checkpoint. 
        string sSystemVariablesCPBuffer;
        int ret = m_poConfig->GetSystemVSM()->GetCheckpointBuffer(sSystemVariablesCPBuffer);
        if (ret == 0)
        {
            // 修改代码   
            // oPaxosMsg.set_systemvariables(sSystemVariablesCPBuffer);
            *(oPaxosMsg.mutable_systemvariables()) = std::move(sSystemVariablesCPBuffer);
        }   

        string sMasterVariablesCPBuffer;
        if (m_poConfig->GetMasterSM() != nullptr)
        {   
            int ret = m_poConfig->GetMasterSM()->GetCheckpointBuffer(sMasterVariablesCPBuffer);
            if (ret == 0)
            {
                // 修改代码   
                // oPaxosMsg.set_mastervariables(sMasterVariablesCPBuffer);
                *(oPaxosMsg.mutable_mastervariables()) = std::move(sMasterVariablesCPBuffer);
            }   
        }   
    }   

    SendMessage(iSendNodeID, oPaxosMsg);
}