Sparkplug - S3 amazonaws com
advertisement
SparkplugSpecification MQTTTopicNamespaceandPayloadDefinition Version1.0 SparkplugSpecificationVersion1.0 Page1 RevisionNumber Date Author Description 1.0 5/26/16 ArlenNipper InitialRelease SparkplugSpecificationVersion1.0 Page2 TableofContents TableofContents.........................................................................................................................................3 TableofFigures............................................................................................................................................6 1. Introduction.........................................................................................................................................7 2. Background..........................................................................................................................................8 3. InfrastructureComponents..................................................................................................................9 3.1. MQTTServer(s)...........................................................................................................................9 3.2. MQTTEdgeofNetwork(EoN)Node...........................................................................................9 3.3. Device.........................................................................................................................................9 3.4. MQTTEnabledDevice(Sparkplug)..............................................................................................9 3.5. MQTTHostNode......................................................................................................................10 3.6. MQTTApplicationNode............................................................................................................10 3.7. Security.....................................................................................................................................10 4. LeveragingStandardsandOpenSource.............................................................................................11 4.1. OASISMQTTV3.1.1Specification.............................................................................................11 4.2. EclipseFoundationIoTResources.............................................................................................11 4.2.1. Paho......................................................................................................................................11 4.2.2. Kura......................................................................................................................................11 4.3. 5. GeneralMessageFlow.......................................................................................................................12 5.1. 6. RaspberryPiHardware.............................................................................................................11 State..........................................................................................................................................12 SparkplugMQTTTopicNamespace....................................................................................................13 6.1. SparkplugTopicNamespaceElements.....................................................................................13 6.1.1. namespaceElement.............................................................................................................13 6.1.2. group_idElement.................................................................................................................13 6.1.3. message_typeElement.........................................................................................................13 6.1.4. edge_node_idElement.........................................................................................................14 6.1.5. device_idElement................................................................................................................14 6.2. 7. ExampleIgnitionFolderStructure............................................................................................14 SparkplugMQTTPayloads..................................................................................................................16 7.1. SparkplugMQTTPayloadEncoding..........................................................................................16 7.1.1. GoogleProtocolBuffers........................................................................................................16 SparkplugSpecificationVersion1.0 Page3 7.1.2. LeveragingExistingKuraProtocolBufferSupport................................................................17 7.2. MQTTEoNBirthandDeathCertificate.....................................................................................18 7.2.1. EoNDeathCertificate(NDEATH)..........................................................................................18 7.2.1.1. Birth/DeathSequenceNumber-bdseq.......................................................................19 7.2.2. EoNBirthCertificate(NBIRTH).............................................................................................19 7.3. 7.2.2.1. Birth/DeathSequenceNumber-bdseq.......................................................................19 7.2.2.2. MessageSequenceNumber-seq................................................................................19 7.2.2.3. UTCMillisecondsTimestamp-timestamp..................................................................20 7.2.2.4. OptionalEoNConfigurationandControlParameters..................................................20 7.2.2.5. IgnitionFolderStructureafteranEoNNodeBirthCertificate.....................................21 MQTTEoNNodeDataPayload(NDATA)..................................................................................22 7.3.1. MessageSequenceNumber-seq........................................................................................23 7.3.2. UTCMillisecondsTimestamp-timestamp...........................................................................23 7.3.3. Dynamicparametervaluesthathavechanged....................................................................23 7.4. MQTTDeviceBirthandDeathCertificate.................................................................................23 7.4.1. MQTTDeviceBirthCertificate(DBIRTH)...............................................................................24 7.4.1.1. MessageSequenceNumber-seq................................................................................24 7.4.1.2. UTCMillisecondsTimestamp-timestamp..................................................................24 7.4.1.3. RequiredDeviceProcessVariableMap.......................................................................24 7.4.1.1. IgnitionTagStructureafterDeviceBIRTHMessage....................................................25 7.4.2. MQTTDeviceDeathCertificate(DDEATH)...........................................................................27 7.5. 7.4.2.1. MessageSequenceNumber-seq................................................................................27 7.4.2.2. UTCMillisecondsTimestamp-timestamp..................................................................27 7.4.2.3. OptionalDeathCertificateParameters........................................................................27 MQTTDeviceDataMessages(DDATA).....................................................................................28 7.5.1. MessageSequenceNumber-seq........................................................................................28 7.5.2. UTCMillisecondsTimestamp-timestamp...........................................................................28 7.5.3. AnyTag/PVvaluethathaschanged.....................................................................................28 7.6. IgnitionGatewayBirthandDeathCertificates.........................................................................29 7.6.1. IgnitionDeathCertificatePayload(STATE)...........................................................................29 7.6.1. IgnitionBirthCertificatePayload..........................................................................................29 7.7. IgnitiontoEoNNodeDataCommand(NCMD).........................................................................30 7.8. IgnitiontoDeviceDataCommand(DCMD)..............................................................................30 SparkplugSpecificationVersion1.0 Page4 8. SparkplugMQTTSessionManagementandMessageFlow...............................................................32 8.1. PrimaryIgnitionSessionEstablishment....................................................................................33 8.2. EoNNodeSessionEstablishment.............................................................................................35 8.3. MQTTDeviceSessionEstablishment........................................................................................37 8.4. GeneralMQTTapplicationsandnon-primaryIgnitionGateway..............................................39 9. SparkplugMQTTDataandCommandMessages...............................................................................40 9.1. EoNNDATAandNCMDMessages............................................................................................40 9.2. DeviceDDATAandDCMDMessages........................................................................................42 10. SparkplugManagementofMultipleMQTTServers.......................................................................45 10.1. MultipleMQTTServerTopology...............................................................................................45 10.2. PrimaryIgnitionGatewaySTATEinMultipleMQTTServerTopologies....................................48 11. SparkplugPersistentversusNon-PersistentConnections..............................................................51 12. GlossaryofTerms...........................................................................................................................52 13. ContactInformation.......................................................................................................................53 SparkplugSpecificationVersion1.0 Page5 TableofFigures Figure1-IgnitionMQTTInfrastructureOverview.......................................................................................7 Figure2-MQTTSCADAInfrastructure.........................................................................................................9 Figure3-SimpleMQTTInfrastructure.......................................................................................................12 Figure4–InitialMQTTEngineTagProviderFolderStructure...................................................................15 Figure5–IgnitionMQTTEngineFolderStructureusingtheSparkplugTopicNamespace.......................15 Figure6-KuraProtocolBufferDefinition..................................................................................................18 Figure7-IgnitionTagStructureafterEoNNodeBIRTH............................................................................22 Figure8-IgnitionTagStructureafterDeviceBIRTHmessage...................................................................26 Figure9-HostSessionEstablishment........................................................................................................33 Figure10-EoNNodeMQTTSessionEstablishment..................................................................................35 Figure11-MQTTDeviceSessionEstablishment.......................................................................................37 Figure12-EoNNodeNDATAandNCMDMessageFlow...........................................................................42 Figure13-DeviceDDATAandDCMDMessageFlow.................................................................................44 Figure14-SingleMQTTServerTopology..................................................................................................45 Figure15-MultipleMQTTServerTopology..............................................................................................46 Figure16-MQTTClienttagsinIgnition.....................................................................................................47 Figure17-EoNMQTTmetrictagsinIgnition............................................................................................48 Figure18-IgnitionSTATEflowdiagram....................................................................................................49 SparkplugSpecificationVersion1.0 Page6 1. Introduction ThisdocumentprovidestheopenandfreelyavailablespecificationforhowEdgeofNetworkorMQTT enabledenddevicescommunicateinMQTTInfrastructureintheInductiveAutomationIgnition ApplicationPlatform.Thedocumentdetailsthestructureandimplementationrequirementsfor compliantMQTTclientdevicestoutilizeCirrusLinksuiteofMQTTModules.Thecomponentsofthe baselineMQTTinfrastructureincludeIgnition,oneormoreMQTTServersandattheCirrusLinkMQTT EngineModule.AdescriptionoftheCirrusLinkMQTTmodulesareasfollows: TheMQTTEngineModule:AddsthefunctionalitytotheIgnitionplatformto bidirectionalcommunicatewithMQTTenablededge-of-the-networkdevicessecurely viaanMQTTServer. TheMQTTDistributorModule:AddsanMQTTservertotheIgnitionplatformthat enablesMQTTclientstosecurelyconnect,publish,andsubscribetodata. TheMQTTTransmission Module: EnablestheTAGswithinIgnitiontobepublishedas MQTTclientdatawithintheMQTTInfrastructure.ItisabridgefromIgnitionTAGsto MQTT. Belowisasamplesolutiondiagram. Figure1-IgnitionMQTTInfrastructureOverview SparkplugSpecificationVersion1.0 Page7 2. Background MQTTwasoriginallydesignedasamessagetransportforRealTimeSCADAsystems.Withinthecontext oftheMQTTspecification,therearemechanismsdefinedtoprovidethestateofanMQTTsessionto provideSCADAHostapplicationsthecurrentstateofanyMQTTdevicenodeintheinfrastructure.The MQTTspecificationdoesnotspecifytheTopicNamespaceorPayloadrepresentationofthedatabeing publishedand/orsubscribed.Thiswasintendedtoprovideflexibilityinfuturesystemimplementations. ThepurposeofthisdocumentistoremaintruetotheoriginalnotionofkeepingtheTopicNamespace andmessagesizestoaminimumwhilestillmakingtheoverallmessagetransactionsbetweenMQTT DeviceNodesandMQTTEngineinstalledontheIgnitionGatewaysimple,efficientandeasyto understandandimplement. SparkplugSpecificationVersion1.0 Page8 3. InfrastructureComponents ThissectiondetailstheinfrastructurecomponentsimplementedwithuseofInductiveAutomation IgnitionPlatformandCirrusLinkMQTTModules. Figure2-MQTTSCADAInfrastructure 3.1. MQTTServer(s) MQTTenabledinfrastructurerequiresoneormoreMQTTServersarepresentintheinfrastructure.The MQTTServercomponentinthearchitectureisrequiredtobecompliantwiththelatestMQTTV3.1.1 specificationandissizedtoproperlymanageallMQTTmessagetraffic. Onecanimplementtheuse(ifrequired)ofmultipleMQTTserversforthepurposeofredundancy,high availability,andscalabilitywithinanygiveninfrastructure. 3.2. MQTTEdgeofNetwork(EoN)Node Inthecontextofthisdocument,anMQTTEdgeofNetwork(EoN)NodeisanyV3.1.1compliantMQTT ClientapplicationthatmanagesanMQTTSessionandprovidesthephysicaland/orlogicalgateway functionsrequiredtoparticipateintheTopicNamespaceandPayloaddefinitionsdescribedinthis document.TheEoNNodeisresponsibleforanylocalprotocolinterfacetoexistinglegacydevices(PLCs, RTUs,FlowComputers,Sensors,etc.)and/oranylocaldiscreteI/O,and/oranylogicalinternalprocess variables(PVs). 3.3. Device TheDevicerepresentsanyphysicalorlogicaldeviceconnectedtotheMQTTEoNNodeprovidingany data,processvariablesormetrics. 3.4. MQTTEnabledDevice(Sparkplug) Thisrepresentsanydevice,sensor,orhardwarethatdirectlyconnectstoMQTTinfrastructureusinga compliantMQTT3.1.1connectionwiththepayloadandtopicnotationasoutlinedinthisSparkplug specification. SparkplugSpecificationVersion1.0 Page9 3.5. MQTTHostNode TheMQTTHostNodeisanyMQTTClientapplicationthatsubscribestoPVmessagesandpublishesPV Commandmessagesdefinedinthisdocument.InmostSCADAinfrastructureimplementations,there willbeonlyonePrimaryMQTTHostNoderesponsibleforthemonitoringandcontrolofagivengroupof MQTTDeviceNodes.ThisdoesnotprecludeanynumberofadditionalMQTTHostNodesparticipatingin theinfrastructurethatareineitherapuremonitoringmode,orintheroleofahotstandbyshouldthe PrimaryMQTTHostNodegooffline. WithintheInductiveAutomationIgnitionplatform,itbecomestheHostNodewheretheMQTTEngine moduleenablesitsMQTTsessionsforsubscribingandpublishingdata.MQTTEoNNodesandassociated DevicesthatfollowthisspecificationareautomaticallyrecognizedbyMQTTEngineandbecomeapart oftheIgnitiontagstructuredynamically.Thisspecificationdetailsthefolderandtagstructurecreated foreachleveloftheTopicNamespace. 3.6. MQTTApplicationNode AMQTTApplicationNodeisanynon-primaryMQTTClientapplicationthatconsumestherealtimePV messagesoranyotherdatabeingpublishedwithproperpermissionandsecurity. 3.7. Security ThereareseverallevelsofsecurityandaccesscontrolconfiguredwithinanMQTTinfrastructure.Froma pureMQTTclientperspective,theclientdoesneedtoprovideauniqueClientID,andanoptional UsernameandPassword.AlthoughaccesscontrolisnotmandatedintheMQTTspecificationforusein MQTTServerimplementations,AccessControlList(ACL)functionalityisavailableformostMQTTServer implementations.TheACLofanMQTTServerimplementationisusedtospecifywhichTopicNamespace anyparticularMQTTClientcansubscribetoandpublishon.Examplesareprovidedonhowtosetupand manageMQTTClientcredentialsandsomeconsiderationsonsettingupproperACL’sontheMQTT Servers. TheMQTTspecificationdoesnotspecifyanyparticularTCP/IPsecurityschemeasitwasenvisagedthat TCP/IPsecuritywould(anddid)changeovertime.AlthoughthisdocumentwillnotspecifyanyTCP/IP securityschemaitwillprovideexamplesonhowtosecureanMQTTinfrastructureusingTLSsecurity. SparkplugSpecificationVersion1.0 Page10 4. LeveragingStandardsandOpenSource TheSparkplugspecificationleveragesasmuchopensourcedevelopmentanddataencodingaspossible asthisisacorebeliefwithinCirrusLinkSolutions. 4.1. OASISMQTTV3.1.1Specification TheSparkplugspecificationspecifiesthatMQTTClientsintheinfrastructureadheretotheMQTTV3.1.1 specification.Thespecificationdocumentationrefersto“mqtt-v3.1.1-os.doc”: http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/mqtt-v3.1.1.html AlsoreferredisanaddendumdocumenttotheMQTTV3.1.1specificationdocumentthatdiscussesbest practicesforimplementingsecurityonMQTTTCP/IPnetworks: http://docs.oasis-open.org/mqtt/mqtt-nist-cybersecurity/v1.0/mqtt-nist-cybersecurity-v1.0.doc 4.2. EclipseFoundationIoTResources TheEclipseFoundationisanexcellentresourceforopensourcesoftwaresupportindustrystandards. WithintheEclipseFoundationisanInternetofThings(IoT)workinggroupprovidingawealthof information. http://iot.eclipse.org/ 4.2.1. Paho PahoisanEclipseFoundationprojectthatoffersexcellentresourcesformature,compliantMQTTClient andMQTTServerimplementationsandwellasadditionalresourcesforallthingsMQTT. http://www.eclipse.org/paho/ 4.2.2. Kura KuraisanotherEclipseFoundationprojectunderIoTresources.KuraisaJava/OSGi-basedframework forIoTgateways,orEoNNodesasdefinedinthisspecification.Kuraalsoprovidesopensource resourcesfortheGoogleProtocolBufferrepresentationofMQTTpayloadsasdefinedinthis specification: https://www.eclipse.org/kura/ 4.3. RaspberryPiHardware ForthesakeofkeepingtheSparkplugspecificationasrealworldaspossible,areference implementationofanEoNNodeandassociatedDeviceisprovidedfortheexamplesandscreenshotsin thisdocument.AllofthiswasimplementedonRaspberryPihardwarerepresentingtheEoNNodewitha PibrellaI/OboardrepresentingtheDevice. SparkplugSpecificationVersion1.0 Page11 5. GeneralMessageFlow ThissectiondiscussesthegenerictopologyshowninFigure2-MQTTSCADAInfrastructureidentifying howeachofthecomponentoftheinfrastructureinteracts. Atthesimplestlevelthereareonlytwocomponentsrequiredasshownbelow.AnMQTTClientandan MQTTServer.Withpropercredentials,anyMQTTClientcanconnecttotheMQTTServerwithoutany notionofotherMQTTClientapplicationsthatareconnected,andcanissuesubscriptionstoany particularMQTTmessage(data)thatitmightbeinterestedinaswellasstartpublishinganydatathatit has.ThisisoneoftheprincipalnotionsofIIoT,thatisthedecouplingintelligentdevicesfromanydirect connectiontoanyoneconsumerapplication. Figure3-SimpleMQTTInfrastructure 5.1. State Inanynetworkarchitecture,networkconnectionStateisimportant.InSCADA,connectionStateis extremelyimportant.StateisthesessionawarenessoftheMQTTEoNandtheMQTTServer.Thevery reasonthatthemajorityofthismarketsectorarestillusinglegacypoll/responseprotocolstomaintaina notionoftheStateoftheconnectionbetweentheSCADAapplicationandtheconnecteddevices.Ipoll,I getaresponse,IknowtheStateofalloftheI/Opoints,butnowImustpollagainbecausethatState mayhavechanged. ManyimplementationsofsolutionsusingMQTTtreatitasasimple,stateless,pub/substatemachine. TheisquiteviableforIoTandsomeIIoTapplications,howeveritisnottakingadvantageofthe capabilityofMQTTbasedinfrastructures. FortheproperimplementationofanyMQTTbasedwithinaSCADAsystem,thebuiltinsessionstate mechanismofMQTTmustbeunderstoodandproperlyimplemented.Ifreal-timecontrolorstateisnot usedforthedataandapplication,stateisnotrequiredandthesystemwilloperatebasedonusingtimestampeddataasLKG(last-known-good)values.(CHECK) OneoftheprimaryapplicationsforMQTTasitwasoriginallydesignedwastoprovidereliableSCADA communicationsoverVSATtopologies.Duetopropagationdelayandcost,itwasnotfeasibletousea poll/responseprotocol.Insteadofapoll/responseprotocolwhereallofthedatawassentinresponseto everypoll,MQTTwasusedto“publish”informationfromremotesitesonlywhenthedatachanged.This techniqueissometimescalledReportbyExceptionorRBE.ButinorderforRBEtoworkproperlyinreal timeSCADA,the“state”oftheenddeviceneedstobeknownatalltimes.Inotherwords,theSCADA hostapplicationslikeIgnitioncouldonlyrelyonRBEdataarrivingreliablyifitcouldbeassuredofthe stateoftheMQTTsession. SparkplugSpecificationVersion1.0 Page12 6. SparkplugMQTTTopicNamespace InordertogetaworkingMessageOrientedMiddleware(MOM)basedSCADAsystemworkingusing MQTT,thefirstthingthatmustbedefinedisaTopicNamespacetoworkwithin.ThebeautyofMQTTis thefactthatyoucanjustcomeupwithanarbitrarytopiclike“Portland/Temperature”,connecttoan MQTTServer,andstartpublishingthetemperaturevalue.Inorderforthisdatatobeusefultoother MQTTClientapplicationsthatwanttoconsumethetemperaturevalues,theTopicNamespaceneedsto beunderstoodbyeveryoneparticipatinginthedataexchange. EveryMQTTmessagepublishedconsistofatopicandapayloadcomponent.Thesecomponentsarethe “overhead”ofanMQTTmessageasmeasuredinbytesonthewire.TheSparkplugspecificationis designedtokeepthesecomponentsmeaningfulandeasytounderstand,butnottogetsoverboseasto negativelyimpactbandwidth/timesensitivedataexchange. 6.1. SparkplugTopicNamespaceElements AllMQTTclientsusingtheSparkplugspecificationwillusethefollowinggeneraltopicstructure: namespace/group_id/message_type/edge_node_id/[device_id] 6.1.1. namespaceElement ThenamespaceelementoftheTopicNamespaceistherootelementthatwilldefineboththestructure oftheremainingnamespaceelementsaswellastheencodingusedfortheassociatedpayloaddata.For thisinitialversionoftheSparkplugspecification,theUTF-8stringconstantforthenamespaceelement willbe: “spAv1.0” ThiselementidentifiesthemessageasbeingaSparkplug“sp”definedmessageusingversion“A”ofthe Sparkplugspecificationencodingschemeversion“1.0”. 6.1.2. group_idElement Thegroup_idelementoftheTopicNamespaceprovidesforalogicalgroupingofMQTTEoNandDevices intotheMQTTServerandbackouttotheconsumingMQTTClients.Theformatofthegroup_idelement isnotdictatedinthatitcanbeanyvalidUTF-8alphanumericStringwiththeexceptionofthereserved charactersof‘+’(plus),‘/’(forwardslash),and‘#’(numbersign).Tominimizeoverheadifthatisagoal ofyourapplication,itshouldbeassmallaspossible.Examplesofwherethe[group_id]mightbeused includeOil/GasapplicationswhereMQTTDevicesonaphysicalpipelinesegmentallhavethesame [group_id].PlantfloorapplicationsmaygroupMQTTDevicesbasedonlogicalcellormanufacturingline requirements. 6.1.3. message_typeElement Themessage_typeelementoftheTopicNamespaceprovidesanindicationastowhattheMQTT Payloadofmessagewillcontain(Note:TheSparkplugMQTTPayloaddefinitionsaredefinedindetail lateristhisdocument).Thefollowingmessage_typeelementsaredefinedfortheSparkplugTopic Namespace: • NBIRTH–BirthcertificateforMQTTEoNNodes. SparkplugSpecificationVersion1.0 Page13 • • • • • • • • NDEATH–DeathcertificateforMQTTEoNNodes. DBIRTH–BirthcertificateforMQTTDevices. DDEATH–DeathcertificateforMQTTDevices. NDATA–Nodedatamessage. DDATA–Devicedatamessage. NCMD–Nodecommandmessage. DCMD–Devicecommandmessage. STATE–Criticalapplicationstatemessage. Thespecificationforeachofthesemessage_typeelementsaredetailedlaterinthisdocument. 6.1.4. edge_node_idElement Theedge_node_idelementoftheSparkplugTopicNamespaceuniquelyidentifiestheMQTTEoNNode withintheinfrastructure.Theformatoftheedge_node_idcanbevalidUTF-8alphanumericStringwith theexceptionofthereservedcharactersof‘+’(plus),‘/’(forwardslash),and‘#’(numbersign).Thetopic elementedge_node_idtravelswitheverymessagepublishedandshouldbeasshortaspossible. 6.1.5. device_idElement Thedevice_idelementoftheSparkplugTopicNamespaceidentifiesanMQTTDeviceattachedtothe MQTTEoNNode.Notethatthedevice_idisanoptionalelementwithintheTopicNamespaceassome messageswillbeeitheroriginatingordestinedtotheedge_node_idandthedevice_idwouldnotbe required.Theformatofthedevice_idisavalidUTF-8alphanumericStringwiththeexceptionofthe reservedcharactersof‘+’(plus),‘/’(forwardslash),and‘#’(numbersign).Thedevice_idmustbeunique fromotherdevicesconnectedtothesameEoNNode,butcanbeduplicatedfromEoNNodetoother EoNNodes. 6.2. ExampleIgnitionFolderStructure UponinstallingtheMQTTEnginemoduleinIgnitionandconfiguringoneormoreavailableMQTT Servers,anew“TagProvider”isaddedtotheIgnitionTagstructurenamed“MQTTEngine”.UntilEdge ofNetworkdevicesjointheMQTTinfrastructure,theMQTTEnginestructureprimarilyprovidesmetrics ontheMQTTEngineinternalMQTTClientsconnectedtothedefinedMQTTServers. SparkplugSpecificationVersion1.0 Page14 Figure4–InitialMQTTEngineTagProviderFolderStructure ButonceanEoNNodeconnectstotheMQTTinfrastructure,MQTTEnginestartsusingtheTopic NamespacedefinedtobuildouttheIgnitionfolderstructurerepresentingEoNNodes,Devices,andthe variousmetric,parameter,andprocessvariablefolders. Figure5–IgnitionMQTTEngineFolderStructureusingtheSparkplugTopicNamespace SparkplugSpecificationVersion1.0 Page15 7. SparkplugMQTTPayloads TheMQTTmessagetransportspecificationdoesNOTdefineanydatapayloadformat,butjustlikeinthe TopicNamespacesectionnamespacedesigndoeshavetotakeplace.Thesameistrueforthedata payloadsbeingpublishedandsubscribedtobytheparticipatingMQTTclientsintheinfrastructure.This sectionoftheSparkplugspecificationdefineshowanMQTTpayloadisencodedandthedatathatis required.NotethatSparkplugcansupportmultiplepayloadsdefinitions.Theinitialreleaseisbasedon theopensourceEclipseKura/GoogleProtobufimplementation. Sparkplugisaddressingtheabsolutefactthatthemajorityofdevicesconnectingintonextgeneration IIoTinfrastructureislegacyequipmentusingpoll/responseprotocols.Thismeanswemusttakein accountregisterbaseddatafromdevicesthattalkprotocolslikeModbus.Theexistinglegacyequipment needstoworkinconcertwithemergingIIoTequipmentthatareabletoleveragemessagetransports likeMQTTnatively. 7.1. SparkplugMQTTPayloadEncoding TheSparkplugspecificationcreatesabandwidthefficientdatatransportforrealtimedevicedata.For WANbasedSCADA/IIoTinfrastructuresthisequatestolowerlatencydataupdateswhileminimizingthe amountoftrafficandthereforecellularand/orVSATbandwidthrequired.ForLANbasedSCADA infrastructures,thisequatestoenablinghigherthroughputofrealtimedatatoconsumerapplications withoutrequiringextremenetworkingtopologiesand/orequipment. ThereisaplethoraofdataencodingtechnologiesavailablethatcanALLbeusedinconjunctionwith MQTT.ConventionalITtoolingprovidesXMLandJSONasthemostpopularencodingtechnologies.But neitheroftheseencodingtechnologiesareefficientinrepresentinglargequantitiesofregisterbased, realtimeprocessvariables.ThesetechnologiesallhavetheirplaceinITdesignstrategies.Sparkplug selectedanexisting,open,andhighlyavailableencodingschemethatefficientlyencodesregisterbased processvariables.TheencodingtechnologyselectedforSparkplugisGoogleProtocolBuffersalso referredtoasGoogleprotobufs. 7.1.1. GoogleProtocolBuffers “ProtocolBuffersareawayofencodingstructureddatainanefficientyetextensibleformat.” GoogleProtocolBuffers,sometimesreferredtoas“Googleprotobufs”,providetheefficiencyofpacked binarydataencodingwhileprovidingthestructurerequiredtomakeiteasytocreate,transmit,and parseregisterbasedprocessvariablesusingastandardsetoftools.GoogleProtocolBuffers developmenttoolsareavailablefor: • • • • • • • C C++ C# Java Python GO JavaScript AdditionalinformationonGoogleProtocolBufferscanbefoundat: SparkplugSpecificationVersion1.0 Page16 https://developers.google.com/protocol-buffers/ 7.1.2. LeveragingExistingKuraProtocolBufferSupport AlthoughtheSparkplugspecificationcouldarbitrarilydefineaGoogleProtocolBufferdefinitionto defineMQTTpayloads,itisthegoalofthisversion“A”ofthespecificationistofollowopensourceand existingstandardswhereverpossible.TheEclipseSoftwareFoundationsponsorsamatureInternetof ThingsGatewayprojectcalledKura(seethetechnicalreferencesectionattheendofthisdocumentfor moreinformation).TheKuraprojecthasalreadydefinedaGoogleProtocolBuffersschemathattargets thetypeofmachineProcessVariableinformationthatwewanttohaveintheSparkplugMQTTmessage payloads.TheProtocolBufferschemacanbefoundatthefollowingURL: https://github.com/eclipse/kura/blob/KURA_1.4.0_RELEASE/kura/org.eclipse.kura.core.cloud/src/main/ protobuf/kurapayload.proto SparkplugSpecificationVersion1.0 Page17 Figure6-KuraProtocolBufferDefinition TheimportantthingtonoteinthisProtocolBufferschemaisthattherequiredparametersarethename andtypeparameters.FormostusecaseapplicationsinSCADA,thinkofthisschemaprovidingan efficientwaytoencodekey:valuepairsofatag(orPLCregisternumber)andanassociatedvalue.The valueenumerationsherecoveralloftherequireddatatypesthatProcessVariablesrequireinclude: • • • • • • • DOUBLE FLOAT INT64 INT32 BOOL STRING BYTES TherearesomemetadatastructuresdefinedwithinthecurrentKuraProtocolBuffersschemathatcan beutilized(optionally)aswell.ThisincludestheINT64timestampthatusedtotimestampUTC millisecondvaluesaswellastheKuraPositionstructureforanyLocationBasedservicesthatmightbe providedbysomeMQTTEoNNodesorDevices. 7.2. MQTTEoNBirthandDeathCertificate AcriticalaspectforMQTTinarealtimeSCADAapplicationismakingsurethattheprimaryMQTTHost Nodeisabletoknowthe“STATE”ofanyEoNand/orDeviceintheinfrastructurewithintheMQTTKeep Aliveperiod(refertosection3.1.2.10intheMQTTSpecification).ToimplementthestateaknownWill TopicandWillMessageisdefinedandspecified.TheWillTopicandWillMessageregisteredinthe MQTTCONNECTsessionestablishment,collectivelymakeupwhatwearecallingtheDeathCertificate. NotethatthedeliveryoftheDeathCertificateuponanyMQTTclientgoingofflineunexpectedlyispart oftheMQTTprotocolspecification,notpartofthisSparkplugspecification(refertosection3.1 CONNECTintheMQTTSpecificationforfurtherdetailsonhowanMQTTSessionisestablishedand maintained). UnliketheDeathCertificatemechanismwhichispartoftheMQTTtransport,theBirthCertificateisa Sparkplugdefinition.TheBirthCertificateisalogicalreciprocaloftheDeathCertificatethatisusedto conveythefactthattheassociateMQTTEoNNodeand/orMQTTDevicenowhasanMQTTsession establishedandcannowstartprovidingrealtimeProcessVariableinformation. ThefirstMQTTmessagethatanEoNNodeMUSTpublishuponthesuccessfulestablishmentofan MQTTSessionisanEoNBIRTHCertificate. 7.2.1. EoNDeathCertificate(NDEATH) TheDeathCertificatetopicforanMQTTEoNNodeis: namespace/group_id/NDEATH/edge_node_id TheDeathCertificatetopicandpayloaddescribedherearenot“published”asanMQTTmessage,but usedasprovidedparameterswithintheMQTTCONNECTcontrolpacketwhenthisMQTTEoNNodefirst establishestheMQTTClientsession. SparkplugSpecificationVersion1.0 Page18 ImmediatelyuponreceptionofanEoNDeathCertification,MQTTEnginewillsettheassociatedIgnition tagdataqualityto“STALE”forALLtagsandparametersassignedtoanyMQTTDeviceconnectedtothis EoNNode. TheMQTTpayloadassociatedwiththistopicwillbeaGoogleProtocolBufferusingtheKuraschema containingthefollowingoptionalparameters: 7.2.1.1. Birth/DeathSequenceNumber-bdseq ‘bdseq’:INT32:[sequence_number] //Birth/DeathSequenceNumber TheBirth/DeathsequencenumberisarequiredSparkplugspecificationfeaturethatisutilizedwhenthe infrastructurecontainsmultipleMQTTServers.ThebdseqmetricvalueMUSTbethesamevalueusedin theassociatedBirthCertificatepayloaddefinedbelow,andMUSTbeanincrementingvalueonall subsequentMQTTsessionestablishments. 7.2.2. EoNBirthCertificate(NBIRTH) TheBirthCertificatetopicforanMQTTEoNNodeis: namespace/group_id/NBIRTH/edge_node_id TheEoNBirthCertificatecontainseverythingrequiredtobuildoutanIgnitiontagtreestructureforall metricsandparametersforthisEoNNode.UponthereceptionofavalidEoNBirthCertificate,MQTT EnginecreatesthetagfolderinformationrequiredtorepresenttheEoNNodeifitdoesnotalready exist,andpopulatetheprovidedmetricinformationintotheappropriatefolders.TheONLINEstateof thisEoNNodewillbesettoTRUEalongwiththeassociatedONLINEDateTimeparameter.Notethatthe EoNBirthCertificateONLYindicatesthenodeitselfisonlineandinanMQTTSession,butanyattached MQTTDeviceswillstillhave“STALE”dataqualityuntilthosedevicescomeonlinewiththeirassociated BirthCertificates. TheMQTTpayloadassociatedwiththistopicwillbeaGoogleProtocolBufferusingtheKuraschema containingthefollowingoptionalparameters: 7.2.2.1. Birth/DeathSequenceNumber-bdseq ‘bdseq’:INT32:[session_number] //Birth/DeathSequenceNumber TheBirth/DeathsequencenumberisarequiredSparkplugspecificationfeatureutilizedwhenthe infrastructurecontainsmultipleMQTTServers.ThebdseqmetricvalueMUSTbethesamevalueusedin theassociatedDeathCertificatepayloaddefinedabove,andMUSTbeanincrementingvalueonall subsequentMQTTsessionestablishments. 7.2.2.2. MessageSequenceNumber-seq ‘seq’:INT32:[message_sequence_number] //MessageSequenceNumber TheMessageSequenceNumberisarequiredparameterthatguaranteespropermessageordering betweenanMQTTEoNNodeandtheSCADA/Hostapplication.ForrealtimesolutionswhereProcess VariablesarebeingpublishedusingReportbyException(RBE),itiscriticalthatallanalogandstate ProcessVariablesarriveinorder.TheMQTTspecificationstatethatallmessagesbedeliveredinthe SparkplugSpecificationVersion1.0 Page19 ordertheyarepublished.ButtheMessageSequenceNumberprovidesanadditionlevelofmonitoring andsafetyininstanceswherethismightnotbethecase.ThevalueoftheMessageSequenceNumber startsatavalueofzero(0)withthisNBIRTHCertificatemessageandincrementoneverysubsequent MQTTmessagesfromthisEoNNodeandeachsubsequentmessageincrementstheMessageSequence Numberbyavalueofexactlyone(1)andincrementtoavalueof255,andthenrolloverto0again.Any primaryMQTTSCADAHostnodemonitorsthereceivedMessageSequenceNumber,andifanoutof ordermessageisdetected,thenALLProcessVariabledatashouldbesetto“STALE”qualityandthe SCADAHostnodewillrequestanewMQTTSessionfromthisEoNNode. 7.2.2.3. UTCMillisecondsTimestamp-timestamp ‘timestamp’:INT64:[current_utc_timestamp] //UTCMillisecondsTimestamp UTCMillisecondsTimestampisanoptionalparameterthatcanbeprovidedintheEoNBirthCertificate. AlthoughanyMQTTHostnodecan(andshould)timestampdataonarrival,UTCtimestampsinthe MQTTmessagescanprovideimportantlatencymetricstomonitortheoverallhealthoftheMQTT infrastructure.InmanySCADAsystemimplementations,allinfrastructurecomponentshaveaccessto thesamemastertimeserver(forexampleusingamasterNTPserver).IncludingtheUTCMillisecond TimestampwitheveryMQTTmessagepublishedallowstheMQTTHostnodetheabilitytoprovidereal timelatencymetricsfromeveryMQTTEoNnodeinthesystem.Thislatencymetricmeasuresthetime forthemessagepublishedfromtheEoNnodetraversingtheinfrastructureandarriveattheSCADA host.ForcellularandVSATbasedsystems,thisisanimportantmetrictokeeptrackof.Itcanalsobe usedtoensurethattherearenoissueswiththemessagesgettingthrutheMQTTServerinfrastructure. 7.2.2.4. OptionalEoNConfigurationandControlParameters EachMQTTEoNNodecanprovideanoptionalnumberofmetricsandparametersforadditional informationaboutthenode.MQTTEngineplacestheseparametersintotheEoNNodetagfolder structurewitheachparameterexposedasanIgnitiontag.ThedifferencebetweentheEoNNode configurationandcontroltagsinthissectionversustheprocessvariabletagsdetailedlaterinthis sectionisthatthesetagsusetheNDATAandNCMDmessagetypesandcanbesegregatedfromphysical devicecommand/controlaccess.ThisisanimportantaspectoftheSparkplugspecificationto understand.Usingpropercredentialandaccessmanagement,usersgainaccesstotheconfiguration parametersofanEoNNodewithouthavingaccesstothephysicaldevicecommand/controlDCMD messages. InformationalparametersliketheHardwareManufactureID,ModelNumber,SoftwareRevision Number,ConfigurationRevisionNumber,etc.canbeprovidedherealongwithanyEoNNodecontrolor configurationparameter.Byusinganappropriateparameterstructurehere,completeconfiguration dashboardscanbebuiltwiththeIgnitiontooling.UsingtheRaspberryPiexamplecode (https://github.com/Cirrus-Link/Sparkplug),thefollowingEoNNodefolderstructureandtagsare exposed: //Rootleveltag ‘UpTimems’:INT64:[upTimeStart] //NodeControlfoldertags ‘NodeControl/Rebirth’:BOOL:false ‘NodeControl/Reboot’:BOOL:false //UpTimeinms //Rebirthcommandcoil //Rebootcommandcoilintothe SparkplugSpecificationVersion1.0 Page20 ‘NodeControl/Nextserver’:BOOL:false //MovetonextMQTTServercommandcoil ‘NodeControl/scan_rate_ms’:INT32:250 //I/Oscanrateinmilliseconds //Propertiesfoldertags ‘Properties/NodeManf’:STRING:’Raspberry’ //Manufacture ‘Properties/HardwareVersion’:STRING:’Pi2ModelB’ //HWVersion ‘Properties/SoftwareVersion’:STRING:’v1.0.0’ //SWVersion ‘Properties/ConfigChangeCount’:INT32:1 //Persistentconfigurationchangecounter NotethatthetagfolderhierarchyasitappearsinIgnitioncanbecontrolledbythestructureoftheBirth Certificatetagnames.Intheexamplegivenabove,the“UpTimems”tagwillappearattherootlevelof theNode.TagsthatcontrolaspectsoftheEoNNodeareplacedintoafoldercalled“NodeControl”. Other,informationonlytagsareshownabovebeingplacedintothe“Properties”folder. TheonlyEoNNodetagstructurethatismandatoryintheSparkplugspecificationisthe“Node Control/Rebirth”nodecontrolBoolean.AnyMQTTClientthatconnectstotheMQTTServer infrastructurewillwanttorequestthatanewBirthCertificateneedstobepublishedinordertolearn aboutthenodeandassociatedDevices.MQTTEnginewillassumethiscontroltagexistsforanyEoN Nodeconnectingintotheinfrastructure. 7.2.2.5. IgnitionFolderStructureafteranEoNNodeBirthCertificate Usingtheactualparametersgivenintheexamplesabove,thereferenceRaspberryPicodecanpublish anEoNNodeBirthCertificateonthefollowingMQTTtopic: spA1.0/SparkplugDevices/NBIRTH/JavaRaspberryPi Uponreceiptofthismessage,MQTTEnginebuildsoutthefollowingfolder/tagstructurewithinIgnition: SparkplugSpecificationVersion1.0 Page21 Figure7-IgnitionTagStructureafterEoNNodeBIRTH AftertheEoNNodeBirthCertificatehasbeenpublished,alloftheparametersdefinedintheBIRTH messagepayloadarenowavailableasIgnitiontags.Theseparametertagshavethesamepropertiesas anyotherIgnitiontagvalue.Asshowninthetagbrowserviewabove,theEoNNodeisonlineand providesspecificinformationaboutthenodethatcanbedisplayedindashboardviewsfortheenduser. Variousnodecommandsarenowavailablelikesendinganewscan_rate_msparametertoreadthe associatedI/Otanewmillisecondperiod.CommandsliketheRebootnodeshownabovecanbe dynamicallybuiltintothisstructureveryquickly.MetricslikethecurrentnodeTCP/IPaddressandthe ConfigurationChangeCountercanbeusedforalarminganddiagnosticswithinIgnition. 7.3. MQTTEoNNodeDataPayload(NDATA) OnceanMQTTEoNNodeisonlinewithproperBirthCertificateswe’reinamodeofquiescentReportby Exception(RBE)ortimebasedreportingofanyparameterinformationthatchanges.Thisenablesthe advantageofthenativeContinuousSessionAwarenessofMQTTtomonitortheSTATEofallconnected MQTTEoNNodedevicesandtorelyonReportbyException(RBE)messagesforanyparameterormetric statechangeovertheMQTTsessionconnection. Asdefinedabove,theDataTopicforanMQTTEoNNodeis: SparkplugSpecificationVersion1.0 Page22 namespace/group_id/NDATA/edge_node_id ThepayloadfortheDatamessageisaGoogleProtocolBufferstructureusingtheKuraschema. 7.3.1. MessageSequenceNumber-seq ‘seq’:INT32:[message_sequence_number] //MessageSequenceNumber TheMessageSequenceNumberisarequiredparameterthatguaranteespropermessageordering betweenanMQTTEoNNodeandtheHostapplication. 7.3.2. UTCMillisecondsTimestamp-timestamp ‘timestamp’:INT64:[current_utc_timestamp] //UTCMillisecondsTimestamp UTCMillisecondsTimestampisanoptionalparameterandifprovidedisthecurrentUTCtimestamp valueatthepointthismessageispublished. 7.3.3. Dynamicparametervaluesthathavechanged Anyparametervaluethathaschangedvalueisaddedtothisstructureinthesameformatusedforthe parameterlistintheEoNBirthCertificateabove: ‘TagName’:ParameterDataType:[CurrentParameterValue] FromtheexampleEoNBirthCertificateabove,anyparametersthatchangevaluesispublishedand placedintothecorrespondingtagsinIgnitionoranyotherinterestedMQTTclientapplication.For example,ifsomeoneweretoupdatetheconfigurationontheexampleRaspberryPiexampleabovethe followingparameterswouldlikelychangevalue. ‘Properties/SoftwareVersion’:STRING:’v1.0.1’ ‘Properties/ConfigChangeCount’:INT32:2 //SWVersionchangedto1.0.1 //Persistentconfigurationchangecounter ThenewvalueswouldbepublishedinaNDATAmessagewiththetopicof: spAv1.0/SparkplugDevice/NDATA/JavaRaspberryPi ThecorrespondingtagvaluesintheParameterfoldershowninFigure7-IgnitionTagStructureafter EoNNodeBIRTHabovewouldbedynamicallyupdatedwiththenewsoftwareversionnumberandthe newconfigurationchangecounter. 7.4. MQTTDeviceBirthandDeathCertificate SparkplugspecifieshowaMQTTEoNNodeusestheMQTTtransportlayerDeathCertificatealongwith theSparkplugdefinedBirthCertificate.MQTTDeviceBirthandDeathcertificatesaredefinedand managedbytheSparkplugspecificationi.e.theyareapplicationlevelpayloadspublishedbytheEoN NodeandarenotpartoftheMQTTtransportspecification. SparkplugSpecificationVersion1.0 Page23 7.4.1. MQTTDeviceBirthCertificate(DBIRTH) TheTopicNamespaceforaBirthCertificateforanMQTTdeviceis: namespace/group_id/DBIRTH/edge_node_id/device_id TheMQTTEoNNodeisresponsibleforthemanagementofallattachedphysicaland/orlogicalMQTT Devices.OncetheEoNNodehaspublisheditsBirthCertificate,MQTTEngineensuresthattheIgnition tagfolderstructurehastheEoNnodeinanONLINEstate.Buteachphysicaland/orlogicaldevice connectedtothisnodewillstillneedtoprovidethisBirthCertificatebeforeMQTTEngine creates/updatesthetagfolderstructure(ifthisisthefirsttimethisdevicehasbeenseen)andsetany associatedProcessVariabletagsinIgnitiontoa“GOOD_DATA”state. 7.4.1.1. MessageSequenceNumber-seq ‘seq’:INT32:[message_sequence_number] //MessageSequenceNumber TheMessageSequenceNumberisaparameterthatguaranteespropermessageorderingbetweenan MQTTEoNNodeandtheHostapplication. 7.4.1.2. UTCMillisecondsTimestamp-timestamp ‘timestamp’:INT64:[current_utc_timestamp] //UTCMillisecondsTimestamp UTCMillisecondsTimestampisanoptionalparameterandifprovidedisthecurrentUTCtimestamp valueatthepointthismessageispublished. 7.4.1.3. RequiredDeviceProcessVariableMap Thefolder/tagstructureforDeviceprocessvariablesandmetricsisidenticaltothewaythatEoNNode parametersarebuilt.ThisisacriticalpartoftheoverallSparkplugspecificationasitprovidesallofthe taginformationandcurrentstatethatMQTTEnginerequirestobuildoutthetagstructureinIgnition andprovidecurrentProcessVariableinformation.EachtagisametricentryintheGoogleProtocol BufferstructureusingtheKuraschema.Eachentryprovidesthefollowingdataforeachtag: ‘TagName’:PVDataType:[CurrentPVValue] Withthisinformation,MQTTEnginecreatesthetagstructurewithinIgnition(ifthisisthefirsttimethis devicehasconnected)foreachtag.NotethatinmostimplementationsthisistheonlytimethatALL tagsarepublished.SinceSparkplugisleveragingthe“ContinuousSessionAwareness”ofMQTTusingthe Birth/DeathCertificates,onceallcurrenttagvaluesarepublished,subsequentdatamessagescansafely reportonlytagvaluesthathavechangedwithintheMQTTsession. TagNamesintheSparkplugspecificationcanbedefinedinasimpleflattagstructurewhereallofthe processvariabletagsappearbelowtheassociatedDeviceinthetaghierarchy.Butinmanycasesit makessensetoorganizethetagsinahierarchicalfolderstructuretomakeiteasiertoviewandmanage thetags.The“TagName”parameterintheBirthCertificatepayloadcanprovideahierarchicaltagpath andMQTTEnginewillcreatetherequiredfolderstructurewithinIgnition.FortheRaspberryPi referenceimplementation,thefollowingdevicefolderstructureandprocessvariabletagsdemonstrates bothaflattagstructureandwellascreatingmorecomplextagpathstobetterorganizetaggroups: //PlacesometagsattherootleveloftheDevicetagstructure ‘button’:BOOL:[currentbuttoninputstate] //PibrellaButtoninput SparkplugSpecificationVersion1.0 Page24 ‘buttoncount’:INT32:[currentcount] //Internalcounterforbuttonpresses ‘buttoncountsetpoint’:INT32:[setpoint] //Read/writesetpointformaxbuttoncount ‘buzzer’:BOOL:false //Pibrellabuzzer //Createafoldercall“Inputs”forthePibrelladigitalinputprocessvariables. ‘Inputs/a’:BOOL:[currentinputstate] //Pibrelladigitalinput_astate ‘Inputs/b’:BOOL:[currentinputstate] //Pibrelladigitalinput_bstate ‘Inputs/c’:BOOL:[currentinputstate] //Pibrelladigitalinput_cstate ‘Inputs/d’:BOOL:[currentinputstate] //Pibrelladigitalinput_dstate //Createafoldercalled“Ouputs”forthePribrelladigitalouputs ‘Outputs/e’:BOOL:[currentoutputstate] //Pibrelladigitaloutput_estate ‘Outputs/f’:BOOL:[currentoutputstate] //Pibrelladigitaloutput_fstate ‘Outputs/g’:BOOL:[currentoutputstate] //Pibrelladigitaloutput_gstate ‘Outputs/h’:BOOL:[currentoutputstate] //Pibrelladigitaloutput_hstate //CreateasubfolderunderOutputscalledLEDsforassociatedLEDcontrol/state ‘Outputs/LEDs/green’:BOOL:[currentLEDstate] //PibrellaGREENLEDstate ‘Outputs/LEDs/red:BOOL:[currentLEDstate] //PibrellaREDLEDstate ‘Outputs/LEDs/yellow:BOOL:[currentLEDstate] //PibrellaYELLOWLEDstate [*Note:ItistheresponsibilityoftheEoNDeviceapplicationto“upcast”unsignedintegervaluestothenextintegerresolution thatIgnitiontagssupport.Forexample,aModbusunsigned16-bitintegervaluewouldbeupcasttoasigned32bitsothat thefullresolutionoftheunsignedintegervaluecanberepresented.If32-bitunsignedintegerprocessvariablesarepresent, accumulatororcountervaluesforexample,thesewouldneedtobe“upcast”to64-bitintegervalues.] ANYfolder/tagvaluecanbepopulatedintheTagMap.Themapaboverepresentsphysicaland calculatedRaspberryPiI/OvaluesbutotherrealtimetagvaluescalculatedbytheMQTTEoNNode connectedtothedevicecanbeaddedtothisstructureaswell. OncetheMQTTDeviceBirthCertificateisdeliveredtoMQTTEngine,thephysical/logicaldeviceis consideredonlineandabletodeliverdiscretetagdataupdatesinrealtimeusingtheDDATAmessage topicandreceivecommandsusingtheDCMDmessage. 7.4.1.1. IgnitionTagStructureafterDeviceBIRTHMessage UsingtheRaspberryPireferenceimplementationswiththeparameterandprocessvariablemappings aboveaDeviceBirthCertificateispublishedonthetopic: spA1.0/SparkplugDevices/DBIRTH/JavaRaspberryPi/Pibrella Uponreceiptofthismessage,MQTTEngineautomaticallybuildsoutthefollowingtagstructurein Ignition: SparkplugSpecificationVersion1.0 Page25 Figure8-IgnitionTagStructureafterDeviceBIRTHmessage OncetheinitialtagstructureisbuiltoutinIgnitionfromtheinformationintheDeviceBirthCertificate, anysubsequentprocessvariablechangesarepublishedusingtheDDATAmessagestructuredetailedin section0below. SparkplugSpecificationVersion1.0 Page26 7.4.2. MQTTDeviceDeathCertificate(DDEATH) Asdefinedabove,theDeathCertificateforanMQTTdeviceis: namespace/group_id/DDEATH/edge_node_id/device_id Intypicalusecasescenarios,itistheresponsibilityoftheMQTTEoNNodetoindicatetherealtimestate ofeitherphysicallegacydeviceusingpoll/responseprotocolsand/orlocallogicaldevices.Regardlessof thedevicebeingmonitoredandprovidingtherealtimetaginformation,intheeventthatanytag informationcouldbebadorquestionable,itistheresponsibilityoftheEoNNodetopublishaDeath Certificateonbehalfoftheenddevice. Formostusecases,thereceptionofaMQTTDeviceDeathcertificateonlyrequirestheoptional parametersoftheMessageSequenceNumber(seq)andtheUTCMillisecondTimestamp(timestamp). 7.4.2.1. MessageSequenceNumber-seq MessageSequenceNumber–‘seq’:INT32:[message_sequence_number] TheMessageSequenceNumberisarequiredparameterthatguaranteespropermessageordering betweenanMQTTEoNNodeandtheHostapplication. 7.4.2.2. UTCMillisecondsTimestamp-timestamp UTCMillisecondsTimestamp–‘timestamp’:INT64:[current_utc_timestamp] UTCMillisecondsTimestampisanoptionalparameterandifprovidedisthecurrentUTCtimestamp valueatthepointthismessageispublished. 7.4.2.3. OptionalDeathCertificateParameters Wheretheusecasedictates,optionalDeathCertificateparameterscanbeprovidedforanyextradetails thatmightberequiredforwhytheDeviceDeathoccurred.Forexample,intypicalusecasescenariosthe legacypoll/responsedevicebeingmonitoredtimedouton“N”numberofpoll/responseattemptsthe MQTTpayloadfortheDeathcouldinclude: ‘death_cert_cause’:STRING:’NoResponse’ //Noresponsefromenddevice or: ‘death_cert_cause’:STRING:’CRCError’ //CRCErrorfromenddevice SparkplugSpecificationVersion1.0 Page27 7.5. MQTTDeviceDataMessages(DDATA) OnceanMQTTEoNNodeandassociatedMQTTDevicesareallonlinewithproperBirthCertificates we’reinamodeofquiescentReportbyException(RBE)reportingofanyPVinformationthatchanges. ThistakesadvantageofthenativeContinuousSessionAwarenessofMQTTtomonitortheSTATEofall connectedMQTTEoNNodedevicesandcanrelyonReportbyException(RBE)messagesforanyPV statechangeovertheMQTTsessionconnection. Asdefinedabove,theDataTopicforanMQTTdeviceis: namespace/group_id/DDATA/edge_node_id/device_id ThepayloadfortheDatamessageisaGoogleProtocolBufferstructureusingtheKuraschema. 7.5.1. MessageSequenceNumber-seq ‘seq’:INT32:[message_sequence_number] //MessageSequenceNumber TheMessageSequenceNumberisarequiredparameterthatguaranteespropermessageordering betweenanMQTTEoNNodeandtheHostapplication. 7.5.2. UTCMillisecondsTimestamp-timestamp ‘timestamp’:INT64:[current_utc_timestamp] //UTCMillisecondsTimestamp UTCMillisecondsTimestampisanoptionalparameterandifprovidedisthecurrentUTCtimestamp valueatthepointthismessageispublished. 7.5.3. AnyTag/PVvaluethathaschanged Anytag/PVvaluethathaschangedvaluecanbeaddedtothisstructureinthesameformatusedforthe PVMapintheDeviceBirthCertificateabove: ‘TagName’:PVDataType:[CurrentPVValue] UsingtheRaspberryPireferenceimplementationexamplegivenabove,ifanyofthePibrellaprocess variablevalueschange,thenaDDATAmessageiscreatedandpublished.Forexample,ifthebuttonon thePibrellaboardwerepressedthenthefollowingmessagewouldbepublished: spA1.0/SparkplugDevice/DDATA/JavaRaspberryPi/Pibrella[‘button’:BOOL:true] AsmanyprocessvariablescanbepublishedasrequiredwithintheDDATAmessagepayload.Thesimple exampleaboveshowsonlythebuttonstateencodedintotheGoogleProtocolBufferpayload.Butifall oftheinputschangedatthesametimethenthepayloadcouldcontainallfouroftheinputstates: ‘Inputs/a’:BOOL:[newstate] ‘Inputs/b’:BOOL:[newstate] ‘Inputs/c’:BOOL:[newstate] ‘Inputs/d’:BOOL:[newstate] spAv1.0/SparkplugDevices/DDATA/JavaRaspberryPi/Pibrella[protobufofallinputstates] SparkplugSpecificationVersion1.0 Page28 7.6. IgnitionGatewayBirthandDeathCertificates IninfrastructureswheremultipleMQTTServersprovideredundancyandscalability,theMQTTEoN Nodesneedtobeawareofthe“state”ofthePrimaryHost.Thisisaccomplishedwithauniquesetof Birth/DeathCertificatesthattheHostMQTTClientMUSTpublishwhenanewMQTTsessionisobtained. InthesamemannerusedfortheMQTTEoNNodeBirth/Deathcertificategeneration,theHost Birth/DeathcertificateswilluseacombinationofthebuiltinMQTTWillTopicandWillPayloadDeath CertificateinconjunctionwithanapplicationlevelBirthCertificatethatispublishedasanMQTT message. ThetopicthataHostnodewilluseisidenticalforboththeBirthandtheDeathcertificate.TheTopic Namespacewillbe: STATE/scada_host_id ItusesanaspectoftheMQTTtransportcalleda“RETAINED”publishinordertomaintainthecurrent stateofthePrimaryHostMQTTClientsessionstatetoallavailableMQTTServers.HowtheHoststateis usedisexplainedindetailinthesectiononSparkplugMQTTSessionManagement. 7.6.1. IgnitionDeathCertificatePayload(STATE) WhentheprimaryIgnitionMQTTclientestablishesanMQTTsessiontotheMQTTServer(s),theDeath CertificatewillbepartoftheWillTopicandWillPayloadregisteredintheMQTTCONNECTtransaction. TheWillTopicasdefinedabovewillbe: STATE/scada_host_id TheWillPayloadwillbetheSTRING“OFFLINE”. TheWillRETAINflagwillbesettoTRUE,andtheWillQoSwillbesetto0. Withtheseparametersset,theMQTTClientfortheHostisanMQTTmessagepublishedonthetopicof scada_host_id/STATE,withastringpayloadof“OFFLINE”anditwillbeaRETAINEDMQTTmessage. 7.6.1. IgnitionBirthCertificatePayload ThefirstmessageaprimaryIgnitionGatewayMUSTpublishisaBirthCertificate.TheIgnitionDeath CertificateisregisteredabovewithintheactualestablishmentoftheMQTTsessionandispublishedasa partofthenativeMQTTtransportiftheMQTTsessionterminatesforanyreason. TheBirthCertificatethatisdefinedhereisanapplicationlevelmessagepublishedbytheHostMQTT Clientapplications. ThetopicusedfortheHostBirthCertificateisidenticaltothetopicusedfortheDeathCertificate: STATE/scada_host_id TheBirthCertificatePayloadistheSTRING“ONLINE”. TheRETAINflagfortheBirthCertificateissettoTRUE,andtheQualityofService(QoS)issetto0. SparkplugSpecificationVersion1.0 Page29 7.7. IgnitiontoEoNNodeDataCommand(NCMD) TheIgnitionGatewaytoEoNNodecommandtopicaboveprovidestheTopicNamespaceusedtosend commandsfromtheIgnitionGatewaytoanyconnectedEoNNodes.FromanMQTTEngineperspective, thismeanssendinganupdatedtagvaluetoanassociatedparametermappedintheEoNBirth Certificateparameterslist. namespace/group_id/NCMD/edge_node_id TheMQTTPayloadisencodedusingthesameGoogleProtocolBuffersKuraSchemadefinedabove. ‘TagName’:PVDataType:[new_PV_value] ThisbecomesaVERYconvenientmechanismforthedisplayofconfigurationparametersandcontrols thatmightbeonvariousEoNdevicesorevenaddingnewonesdynamically.Intheexamplegivenabove fortheEoNNodeBirthCertificate,someEoNNodecontrolsprovidedcannowbewrittentousingthis message.Byclickingonthe‘Nextserver’coilintheIgnitiontagstructure,aKuraencodedpayloadis publishedtocausetheEoNNodetodisconnectfromthecurrentMQTTServerandwalktothenextone initsconfigurationtableshowninthefollowingstructureoftopic[payload]: //SendcommandtowalktonextavailMQTTServer spAv1.0/Group_001/NCMD/EoN_001[‘Nextserver’:BOOL:true] AnytimeanewMQTTApplicationclientjoinstheMQTTinfrastructureitmightwanttoseeacompletely newsetofBirthCertificatesfromthisEoNNodetogetinsyncwithallavailabledata.Usingtheproper group_idanddevice_idinthetopicstructure,acommandmessageissenttocausetheEoNNodeto reissueallBirthCertificateinformationas: //SendcommandtoReissueallEoNandDeviceBirthCerts spvA1.0/Group_001/NCMD/EoN_001[‘Rebirth’:BOOL:true] WithproperpermissionsinIgnition,thesystemcouldwritetotheRebootcoilmappedoutinthe exampleabovetocausetheremoveEoNNodetoreboot: //RebootthetargetEoNNode spvA1.0/Group_001/NCMD/EoN_001[‘Reboot’:BOOL:true] 7.8. IgnitiontoDeviceDataCommand(DCMD) TheIgnitionGatewaytoDevicecommandtopicaboveprovidestheTopicNamespaceusedtosend commandsfromIgnitiontoanyconnectedDevice.FromanMQTTEngineperspective,thismeans sendinganewtagvaluetoanassociatedparametermappedintheDeviceBirthCertificateparameters ortaglist. namespace/group_id/DCMD/edge_node_id/device_id TheMQTTPayloadisencodedusingthesameGoogleProtocolBuffersKuraSchemadefinedabove. ‘TagName’:PVDataType:[new_PV_value] SparkplugSpecificationVersion1.0 Page30 UsingtheMQTTDevicetagsmapprovidedintheDeviceBirthCertificateexampleabove,commandscan beissuedtoanywritablePVwithintheIgnitiontagstructure: //SendacommandtoturntheGREENLEDOn spAv1.0/SparkplugDevices/DCMD/JavaRaspberryPi/Pibrella[‘Outputs/LEDs/green’:BOOL:true] SparkplugSpecificationVersion1.0 Page31 8. SparkplugMQTTSessionManagementandMessageFlow AnMQTTbasedSCADAsystemisuniqueinthattheHostnodeisNOTresponsibleforestablishingand maintainingconnectionstothedevicesasisthecaseinmostexistinglegacypoll/responsedevice protocols.WithanMQTTbasedSCADAsystem,boththeHostapplicationaswellasthedevicesestablish MQTTSessionswithacentralMQTTServerorSevers.Thisisthedesiredfunctionalityasitprovidesthe necessarydecouplingfromanyoneapplicationandanygivendevice.AdditionalMQTTclientscan connectandsubscribetoanyoftherealtimedatawithoutimpactingtheprimarySCADAHost application. DuetothenatureofrealtimeSCADAsolutions,itisveryimportantfortheprimarySCADAHostandall connectedMQTTEoNNodestohavetheMQTTSessionSTATEinformationforeachother.Inorderto accomplishthistheSparkplugTopicNamespacedefinitionsforBirth/Deathcertificatesalongwiththe definedpayloadsprovidebothstateandcontextbetweentheSCADAHostMQTTclientandthe associateddevicesideMQTTClients.Inmostusecasesandsolutionscenariostherearetwoprimary reasonsforthis“designation”ofaprimarySCADAHost: 1. OnlythePrimaryIgnitionHostshouldhavethepermissiontoissuecommandstoMQTTDevices. 2. InhighavailabilityandredundancyusecaseswheremultipleMQTTServersareused,MQTTEoN NodesneedtobeawareofwhetherIgnitionhasnetworkconnectivitytoeachandeveryMQTT Serverintheinfrastructure.IfthePrimaryIgnitionSTATEshowsthatanEoNNodeisconnected toanMQTTServerthatthePrimaryIgnitionGatewayisNOTconnectedto,thentheEoNNode shouldwalktothenextavailableMQTTServerwhereSTATEfortheIgnitionGatewayis ‘ONLINE’. SparkplugSpecificationVersion1.0 Page32 8.1. PrimaryIgnitionSessionEstablishment OncetheMQTTEnginemoduleisinstalledinIgnition,theIgnitionGatewayConsoleprovidesanew MQTTEngineSettingstabinConfigurationàMQTTEngineàSettings.Thisconfigurationmenuallows forthedefinitionofoneormoreMQTTServersthatarepresentintheinfrastructureaswellasifthis IgnitioninstanceisaPRIMARYinstanceintheinfrastructure(Note:ForIgnitioninstancesthatarenot primary,refertosection8.4,GeneralMQTTapplicationsandnon-primaryIgnitionGateway.below). OncetheMQTTEnginemoduleisinstalledandconfigured,itwillimmediatelytrytocreateaHostMQTT SessionwiththeconfiguredMQTTServerinfrastructure.NotethattheestablishmentofanMQTTHost NodesessionisasynchronousofanyotherMQTTClientsession.IfEoNnodesarealreadyconnectedto theMQTTServerinfrastructure,MQTTEnginewillbeabletosynchronizewiththem.IfassociatedEoN nodesarenotconnected,MQTTEnginewillregisterthemwhentheypublishtheirBirthCertificate. Figure9-HostSessionEstablishment ThesessiondiagraminFigure9-HostSessionEstablishmentshowsaverysimpletopologywithasingle MQTTServer.Thestepsoutlinedinthesessiondiagramaredefinedasfollows: 1. MQTTEnginewilltrytocreateanMQTTSessionusingtheMQTTCONNECTControlPacket(refer tosection3.1intheMQTTV3.1.1specification).ADeathCertificateisconstructedintotheWill TopicandWillPayloadoftheoftheConnectControlPacketwithaWillQoS=0andWillRetain= true.TheMQTTCONNECTControlPacketisacknowledgedassuccessfulwithavalidCONNACK ControlPacket.Fromthispointforwardintime,theMQTTServerisreadytodeliveraHost DeathCertificateanytimetheMQTTEngineMQTTClientlosesTCP/IPconnectivitytotheMQTT Server. SparkplugSpecificationVersion1.0 Page33 2. OnceanMQTTSessionhasbeenestablished,MQTTEnginewillpublishanewSTATEmessageas definedininsection7.6.1,IgnitionBirthCertificatePayload.AtthispointMQTTEnginecan updatetheMQTTClientmetrictagsinIgnitionwithacurrentstateofONLINE. 3. WiththeMQTTSessionestablished,andaSTATEBirthCertificatepublished,MQTTEnginewill issueanMQTTsubscriptionforthedefinedSparkplugTopicNamespace,“spv1.0/#”.MQTT EngineisnowreadytostartreceivingMQTTmessagesfromanyconnectedEoNnodewithinthe infrastructure.SinceMQTTEngineisalsorelyingontheMQTTSessiontotheMQTTServer(s), theavailabilityofServerstoIgnitionisalsobeingmonitoredandreflectedintheMQTTClient metricstagfolderinIgnition. 4. IfatanypointintimeMQTTEnginelosesTCP/IPconnectivitywiththedefinedMQTTServer(s), theONLINEstateoftheServerisimmediatelyreflectedintheMQTTClientmetricstagfolderin Ignition.AlltagdataassociatedwithanyMQTTEoNNodethatwasconnectedtothatMQTT Serverwillupupdatedtoa‘STALE’dataquality. SparkplugSpecificationVersion1.0 Page34 8.2. EoNNodeSessionEstablishment AnyEoNNodeintheMQTTinfrastructuremustestablishanMQTTSessionpriortoproviding informationforconnectedMQTTDevicenodes.MostimplementationsofanMQTTEoNNodeforreal timeSCADAwilltrytomaintainapersistentMQTTSessionwiththeMQTTServerinfrastructure.But thereareusecaseswheretheMQTTSessiondoesnotneedtobepersistent.Ineithercase,anEoN NodecantrytoestablishanMQTTsessionatanytimeandiscompletelyasynchronousfromanyother MQTTClientintheinfrastructure.Theonlyexceptiontothisruleistheusecasewherethereare multipleMQTTServersandaPrimaryHostapplication. Figure10-EoNNodeMQTTSessionEstablishment ThesessiondiagraminFigure10-EoNNodeMQTTSessionEstablishmentshowsaverysimpletopology withasingleMQTTServer.Thestepsoutlinedinthesessiondiagramaredefinedasfollows: 1. TheEoNNodeMQTTclientwillattempttocreateanMQTTsessiontotheavailableMQTT Server(s)usingtheMQTTCONNECTControlPacket(refertosection3.1intheMQTTV3.1.1 specification).TheDeathCertificateconstructedintotheWillTopicandWillPayloadfollowsthe formatdefinedinsection7.2.1,EoNDeathCertificate(NDEATH).TheMQTTCONNECTControl PacketisacknowledgedassuccessfulwithavalidCONNACKControlPacket.Fromthispoint forwardintime,theMQTTServerisreadytodeliveranEoNNodeDeathCertificatetoany subscribingMQTTClientanytimeTCP/IPconnectivityislost. 2. OnceanMQTTSessionhasbeenestablished,theEoNNodeMQTTclientwillpublishan applicationlevelBirthCertificateasdefinedinsection7.2.2,EoNBirthCertificate(NBIRTH).At thispointMQTTEnginewillhavealloftheinformationrequiredtobuildouttheEoNNode metricsandparametersintoanassociatedIgnitiontagfolderstructureandshowtheEoNNode inan“ONLINE”state. SparkplugSpecificationVersion1.0 Page35 3. AfterpublishingtheEoNNodeBirthCertificate,thelastthingtheEoNNodeMQTTclientneeds todoistoissueasubscriptiontospecificSparkplugdefinedtopics.ThesubscriptiontoNCMD leveltopicsensuresthatEoNtargetedmessagesfromMQTTEnginearedelivered.The subscriptiontoDCMDensuresthatDevicetargetedmessagesfromMQTTEnginearedelivered. InapplicationswithmultipleMQTTServersanddesignatedPrimaryHostapplications,the subscriptiontoSTATEinformstheEoNNodethecurrentstateofthePrimarySCADAHost.At thispointtheEoNNodehasfullycompletedthestepsrequiredforestablishingavalidMQTT SessionwithMQTTEngineandIgnition. 4. IfatanypointintimetheEoNNodeMQTTClientlosesTCP/IPconnectivitytothedefinedMQTT Server(s),aDeathCertificateisissuebytheMQTTServeronbehalfoftheEoNNode.Upon receiptoftheDeathCertificate,MQTTEnginewillsetthestateoftheEoNNodeto‘OFFLINE’ andupdatealltimestampmetricsconcerningtheconnection.Anydefinedmetricandparameter tagswillbesettoa‘STALE’dataquality. SparkplugSpecificationVersion1.0 Page36 8.3. MQTTDeviceSessionEstablishment TheSparkplugspecificationisprovidedtogetrealtimeprocessvariableinformationfromexistingand newenddevicesmeasuring,monitoringandcontrollingaphysicalprocessintoandMQTTMOM infrastructureandtheIgnitionIndustrialInternetofThingsapplicationplatform.Inthecontextofthis documentanMQTTdevicecanrepresentanythingfromexistinglegacypoll/responsedrivenPLCs,RTUs, HARTSmartTransmitter,etc.,tonewgenerationautomationandinstrumentationdevicesthatcan implementaconformantMQTTclientnatively. TheprecedingsectionsinthisdocumentdetailhowMQTTEngineinteractswiththeMQTTServer infrastructureandhowthatinfrastructureinteractswiththenotionofanMQTTEoNNode.Buttoa largeextentthetechnicalrequirementsofthosepiecesoftheinfrastructurehavealreadybeen provided.Formostusecasesinthismarketsectortheprimaryfocuswillbeontheimplementationof theSparkplugspecificationbetweenthenativedeviceandtheEoNNodeAPI’s. Inordertoexposeandpopulatethemetric,parameter,andprocessvariabletaginformationfromany intelligentdevice,thefollowingsimplesessiondiagramoutlinestherequirements: Figure11-MQTTDeviceSessionEstablishment ThesessiondiagraminFigure11-MQTTDeviceSessionEstablishmentshowsasimpletopologywithall oftheSparkplugelementsinplacei.e.Ignition,MQTTEngine,MQTTServer(s),MQTTEoNNodeandthis element,theMQTTDeviceelement.Thestepsoutlinedinthesessiondiagramaredefinedasfollows: 1. ThisflowdiagramassumesthatatleastoneMQTTServerisavailableandoperationalwithinthe infrastructure.WithoutatleastasingleMQTTServertheremainderoftheinfrastructureis unavailable. 2. TheSessionEstablishmentofIgnitionusingtheMQTTEnginemoduleisdescribedinsection8.1, PrimaryIgnitionSessionEstablishment.TheestablishmentofthissessionisactuallyVERY arbitrarybasedonusecaseasEoNNodescanandwillestablishsessionswithMQTT infrastructureswithorwithoutthiscomponentacross“N”numberofHostapplications. 3. TheSessionEstablishmentoftheassociatedMQTTEoNNodeisdescribedinsection8.2,EoN NodeSessionEstablishment.ThisflowdiagramassumesthattheEoNNodesessionhasalready beenestablishedwithMQTTEngine. SparkplugSpecificationVersion1.0 Page37 Dependingonthetargetplatform,theEoNNodemaybeaphysical“EdgeofNetwork”gatewaydevice devicepollingphysicallegacydevicesviaModbus,AB,DNP3.0,HART,etc.,aMQTTenabledsensoror sensorordevice,oritmightbealogicalimplementationofoneoftheCirrusLinkreference implementationsforprototypeEoNNodesrunningontheRaspberryPIplatform.Regardlessofthe theimplementation,atsomepointthedeviceinterfacewillneedtoprovideastateandassociated associatedmetrics,parameters,andprocessvariabletopublishtotheMQTTinfrastructure.State#4in State#4inthesessiondiagramrepresentsthestateatwhichthedeviceisreadytoreportallofits itsmetadataandprocessvariableinformationtotheMQTTEoNNodeasdefinedinSparkplug.Itisthe istheresponsibilityoftheEoNNode(logicalorphysical)toputthisinformationinaformdefinedin0, definedin0, SparkplugSpecificationVersion1.0 Page38 4. MQTTDeviceBirthCertificate(DBIRTH).UponreceivingtheDBIRTHmessage,MQTTEnginecan buildoutthepropermetric,parameter,andtaginformationinIgnition. FollowingtheSparkplugspecificationinsection0, SparkplugSpecificationVersion1.0 Page39 5. MQTTDeviceDataMessages(DDATA),allsubsequentmetric,parameter,andprocessvariable informationarepublishedtoMQTTEngineonaReportbyException(RBE)basisusingthe DDATAmessageformat. 6. InatanytimetheMQTTDevice(logicalorphysical)cannotproviderealtimeinformation,the MQTTEoNNodespecificationrequiresthatanMQTTDeviceDeathCertificatebepublished.This willinformMQTTEnginethatallmetric,parameter,andprocessvariableinformationinIgnition besettoa‘STALE’dataquality. SparkplugSpecificationVersion1.0 Page40 8.4. GeneralMQTTapplicationsandnon-primaryIgnitionGateway. Asnotedabove,thereisthenotionofaPrimaryIgnitionGatewayinstanceintheinfrastructurethathas therequiredpermissionstosendcommandtoDevicesandthefactthatallEoNNodesneedtobeaware ofthefactthatthePrimaryIgnitionGatewayisconnectedtothesameMQTTServeritsconnectedtoor itneedstowalktoanotheroneintheinfrastructure.Butbothoftheseareknownrequirementsofa missioncriticalSCADAsystem. ButunlikelegacySCADAsystemimplementations,allrealtimeprocessvariableinformationbeing publishedthrutheMQTTinfrastructureisavailabletoanynumberofadditionalMQTTClientsinthe businessthatmightbeinterestedinsubsetsifnotalloftherealtimedata. TheONLYdifferencebetweenaPrimaryIgnitionMQTTclientandallotherclientsthatnon-primary ClientdoNOTissuetheSTATEBirth/Deathcertificates. SparkplugSpecificationVersion1.0 Page41 9. SparkplugMQTTDataandCommandMessages Lookingbackinthisdocumentwe’vedescribedthefollowingcomponents: • • • • • • • • • • • IgnitionGateway MQTTEngine MQTTServer(s) EdgeofNetwork(EoN)Nodes Devices TopicNamespace PayloadEncoding BirthCertificates DeathCertificates STATEMessages Ignition,EoNNode,andDeviceSessionEstablishment AllofthesespecificationsanddefinitionsgettotheprimarygoalofSparkplug,thatistodeliverarichset ofrealtimeDeviceprocessvariabledataextremelyefficientlytomanydataconsumerswithinthe EnterprisewhilestillprovidingabestinclassCommand/ControlSCADAsystem. ThedisruptivenotionoftheemergingIIoTmindsetisthatintelligentdevicesshouldbesmartenoughto deliverprocessvariabledatatotheinfrastructurewhenitisrequired.Butthefactofthematteristhat theexistingpopulationof100’sofmillionsofthesmartdevicesneedtobe“asked”ifsomethinghas changedusingpoll/responseprotocols.Thisiswhywe’reseeingtheemergenceofedgedevices throughouttheindustrialsector.Forthedecadeormorethatitwilltakefordevicemanufacturesto embedIIoTtechnologynatively,thesolutionbeingemployedtodayistoplacethiscapabilityinsmall embeddeddevicesclosertothedataproducersthemselves.SowithintheSparkplugspecificationthese devicescalledEdgeofNetworkNodes(EoN)representthisnewclassofGateway,EdgeController,Edge ofNetworkNode,ProtocolGateway,andmanymoreacronymsforthesameclassofdevices.The capabilityofthesedevicesareinanextremerangeoflowpowermicrocontrollerstomulticoreInteland ARMbasedprocessors.TheoperatingsystemsrangefromfullembeddedLinuxkernelsandWindows embeddedtosmallbaremetalRTOS’s.Regardlessofthecategorythesegatewaydevicesfallinto,the simplicity,ofMQTTandtheSparkplugspecificationshouldbeavailableacrosstheboard. ThissectionoftheSparkplugspecificationgoesintodetailonhowmetric,parameter,andprocess variabledataarepublished/subscribewithinanMQTTinfrastructureinrealtimeandtheresultingtag informationthatIgnitioncanread/writeto. 9.1. EoNNDATAandNCMDMessages We’llstartthissectionwithadescriptionofhowmetricandparameterdataispublishedtoIgnitionfrom anEoNNodeintheMQTTinfrastructure.ThedefinitionofanEoNNodeisgenericinthatitcan representbothphysical“EdgeofNetworkGateway”devicesthatareinterfacingwithexistinglegacy equipmentandalogicalMQTTendpointfordevicesthatnativelyimplementtheSparkplugspecification. SectionXXXXXabovedefinestheBirthCertificateMQTTPayloadandthefactthatitcanprovideany numberofmetricandparametertagsthatwillbeexposedinIgnition.Someofthesetagswillbe“read only”suchas: SparkplugSpecificationVersion1.0 Page42 • • • • • • • • • EoNManufacture EoNDeviceType EoNSerialNumber EoNSoftwareVersionNumber EoNConfigurationChangeCount EoNPosition(ifGPSdeviceisavailable) EoNCellularRSSIvalue(ifcellularisbeingused) EoNPowerSupplyvoltagelevel EoNTemperature Othermetricsmaybedynamicand“read/write”tagssuchas: • • • • EoNRebirthcommandtorepublishallEoNandDeviceBirthCertificates. EoNNextservercommandtomovetonextavailableMQTTServer. EoNRebootcommandtoreboottheEoNNode. EoNPrimaryNetwork(PRI_NETWORK)where1=Cellular,2=Ethernet TheimportantpointtorealizeisthatthetagsexposedinIgnitionforuseinthedesignofapplications arecompletelydeterminedbywhatmetricandparametervaluesarepublishedintheEoNBirth Certificate.EachspecificEoNNodecanbestdeterminewhatdatatoexpose,andhowtoexposeit,and itwillautomaticallyappearintheIgnitiontagstructure.Parameterscanevenbeaddeddynamicallyat runtimeandwithanewBirthCertificate,theseparameterswillautomaticallybeaddedtotheIgnition tagstructure. TheotherVERYimportantdistinctiontomakehereisthatEoNNodeNDATAandNCMDmessagesare decoupledfromtheDevicelevelcommandandcontrolmessagesofDDATAandDCMD.Thisdecoupling intheTopicNamespaceisimportantbecauseitallowsinteractionfromallMQTTClientsinthesystem (tothelevelofpermissionandapplication)withtheEoNNodes,butNOTtothelevelofsendingDevice commands.MQTTEngineprovidesaconfigurationparameterthatwillBLOCKoutputDDATAandDCMD messagesbutstillallowNDATAandNCMDmessagestoflow.Inthismanner,multipleIgnitionsystems canbeconnectedtothesameMQTTinfrastructure,butonlytheoneswithDevicecommandsenabled canpublishDevicecommands. Thefollowingsimplemessageflowdiagramdemonstratesthemessagesusedtoupdateachanging cellularRSSIvalueinIgnitionandsendingacommandfromIgnitiontotheEoNNodetouseadifferent primarynetworkpath. SparkplugSpecificationVersion1.0 Page43 Figure12-EoNNodeNDATAandNCMDMessageFlow 1. AssumingMQTTServerisavailable. 2. AssumingthatMQTTEngineasanestablishedMQTTSessionwiththeMQTTServer(s). 3. TheEoNNodehasanestablishedMQTTSessionandtheBirthCertificatehasbeenpublished. Ignitionnowhasalldefinedmetricandparametertagsandtheircurrentvalue. 4. TheEoNNodeismonitoringitslocalcellularRSSIlevel.ThelevelhaschangedandnowtheEoN NodewantstopublishthenewvaluetotheassociatedtaginIgnition. 5. Fromandoperationalrequirement,theEoNNodeneedstobetoldtoswitchitsprimary networkinterfacefromcellulartoEthernet.FromIgnitionthenewvalueiswrittentothetag andMQTTEnginewillautomaticallypublishthenewvaluetotheEoNNodeparameters. 9.2. DeviceDDATAandDCMDMessages TheultimateintentoftheSparkplugspecificationistofacilitatethepublishingofrealtimeprocess variabledatafromexistingdevicesinthefieldorontheplantfloorintoaMessageOrientedMiddleware infrastructure.ItisrecognizedthatthisrepresentsaHUGEplethoraofdevicesacrossthespectrumin theIndustrialdevicemarketsectorspanningdecadesofinstalledandlegacyinfrastructure.Throughthe useofMQTTandtheSparkplugspecification,Ignitioncanbecometheultimate“SwissArmyKnife”as theIndustrialApplicationDevelopmentplatformfortheemergingIIoTinfrastructures. ThereasonthatthenotionofanEoNNodeispresentinSparkplugisarealizationthatifexistingsmart deviceinfrastructurecannotbeaddressedwithemergingtoolsanddevelopmentplatforms,the migrationfromlegacymethodologies,protocols,infrastructures,anddatasiloswillneveroccurandthe envisagedbenefitsofFactory4.0willnevercometofruition.EoNNodescanrepresentphysical hardwareintheplantorinthefieldthatcanaddressconvertinglegacypoll/responseprotocols(while providingtheoftenrequirednetworksecurityatthesametime)intorealtimeMQTTmessages.Atthe sametime,EoNNodescanrepresentalogicalsoftwareagentinnextgenerationdevicesthatcan nativelyproviderealtimeprocessvariablesviaMQTT.ByusingGoogleProtocolBuffersandtheKura SparkplugSpecificationVersion1.0 Page44 schemainconcertwiththeIgnitiontagstructure,devicescanbedecoupledfromapplicationsand valuabledeviceinformationcanbemuchmoreeasilysharedwithintheenterprise. ButultimatelytheEoNNodesareonlyinplacetosupporttheultimatedataproducer/consumer,the realdevicesontheplantfloororinthefield.TheSparkplugspecificationprovidesawaytonamethe processvariablesacquiredfromthesedevicesalongwithdefiningtheappropriatedatatypeandcurrent value.OncetheprocessvariablesarepublishedintoanMQTTinfrastructure,theycanbeconsumedby anynumberofapplicationsareareinterestedinthevalues,includingoneormoreIgnitionGateway instances. Section0, SparkplugSpecificationVersion1.0 Page45 MQTTDeviceBirthCertificate(DBIRTH)abovedefinestheDeviceBirthCertificateMQTTPayloadthat definesalloftheprocessvariablesbeingprovidedbytheassociateddevice.Uponthereceiptofthe BirthCertificate,MQTTEnginebuildsouttheentireDevicefolderundertheassociatedEoNNodeand createsallofthedefinedtagswithcurrentvaluesandstates.Thetagnamingconventionisleftentirely totheimplementationoftheSparkplugspecificationontheEoNNode.Forexample,ifanEoNNodewas usingtheModbusprotocoltopollaPLClocally,thenthetagnamesusedforIgnitionmightremainthe conventionalModbusregisteraddressesof0xxxx,1xxxx,3xxxx,and4xxxx.ConverselytheEoNNode mightprovidethecapabilityofassigningmoremeaningfultagnamestotheModbusregisterprocess variablessuchasSuctionPressure,TankLevel,ProcessTemp,etc.AnEoNNodeprovidingtheinterface toHARTenables4-20maSmartTransmitterscouldusetheactualHARTassignedprocessvariables namessuchasPV1,PV2,LRV,URV,etc. Regardlessofthetagnamingconventionused,uponreceiptoftheDeviceBirthCertificate,MQTT EnginewillcreatetheprocessvariabletagstructurewithinIgnitionwheretheywillbeimmediately usablebytheIgnitionplatform.Inthefollowingmessageflowdiagram,we’llusetheexampleofa ModbusPLC.ThisexamplewillalsoassumethattheEoNNodeconnectedtothePLCcanprovide descriptivetagnamestoeachoftheModbusregisteraddresses: • • • • • • • ValveOpen ValveClose ValveLS1 ValveLS2 SuctionPressure DischargePressure Setpoint //mapModbusCoil#1totheValveOpencommand. //mapModbusCoil#2totheValveClosecommand. //mapModbusStatusInput10,001totheValveLimitSwitch#1 //mapModbusStatusInput10,002totheValveLimitSwitch#2 //mapModbusInputRegister30,001totheSuctionPressure //mapModbusInputRegister30,002totheDischargePressure //mapModbusHoldingRegister40,001tothesetpointvalue SparkplugSpecificationVersion1.0 Page46 Figure13-DeviceDDATAandDCMDMessageFlow 1. 2. 3. 4. 5. 6. 7. 8. AssumingMQTTServerisavailable. AssumingthatMQTTEnginehasanestablishedMQTTSessionwiththeMQTTServer(s). AssumingthattheEoNNodehasanestablishedMQTTSessionwiththeMQTTServer(s). UponreceivingtheDeviceBirthCertificate,MQTTEnginecreates/updatestheDevicefolderin IgnitionandallassociatedtagsdefinedforthisDevice. TheEoNNodeispollingtheModbusPLClocallyanddetectsthattheSuctionPressurein ModbusRegister30,001haschangedvalues.ThisisimmediatelyplacedintothedefinedDevice MQTTPayloadandpublished.Uponreceipt,MQTTEngineupdatestheSuctionPressuretag value. The‘ValveOpen’BooleaniscommandedonanIgnitiondashboard.MQTTEnginecreatesthe DCMDpayloadrequiredbytheEoNNodetosendaModbusForceCoilCommandtoCoil#1. TheValveOpencommandcausestheMotorOperatedvaluetostartmovingwhichinturn changesthestateofthetwoMOVlimitswitches.ThenextModbuspollresultsinboth‘Valve LS1’and‘ValveLS2’inchangingstate.BothchangesareplacedintoaDDATAMQTTpayloadand published.MQTTEnginereceivestheDDATAmessageandimmediatelyupdatestheassociated tagvaluesinIgnition. OnthenextModbuspoll,theDischargePressurechangesvalue.Thenewvalueispublishedto theMQTTServer.MQTTEnginereceivesthemessagesandimmediatelyupdatestheDischarge Pressuretag. Thevalvethatwascommandedin#6abovefinallygoesfroman“intransit”statetofullyopen. ThiscausesbothlimitswitchinputstothePLCtochangestate.TheEoNNodepicksupthisstate change,formatsaDDATAmessagewiththenewvaluesandpublishesittotheMQTTServer. MQTTEnginereceivesthemessageandimmediatelyupdatestheassociatedtagsinIgnition. SparkplugSpecificationVersion1.0 Page47 10. SparkplugManagementofMultipleMQTTServers TherearemanyinstanceswheretheMQTTinfrastructurewillcontainmultipleMQTTServers.Thereare severalreasonwhymultipleMQTTServersareutilized.Certainly,applicationswhereredundancy,high availability,anddisasterrecoveryrequiremultipleMQTTservers.MultipleMQTTServerscanalsobe deployedwherescalabilityacrossalargenumberofMQTTEoNNodesisrequired.Otherhybridmodels useonpremiseMQTTserversastheprimaryandCloudbasedMQTTServerand/orServicesas secondaryandtertiary. RegardlessofthereasonformultipleMQTTServers,thereareseveralarchitecturalconsiderationsthat needtobeaddressedbytheSparkplugspecification. 10.1. MultipleMQTTServerTopology Foracomparisonofthetopologies,thefollowingdiagramshowsatypicalinfrastructurewithasingle MQTTServerinstance: Figure14-SingleMQTTServerTopology ThesingleMQTTServertopologyshownaboveisaperfectlyviablesolutioninmanyinstances.Butit doesrequireahighlyavailableMQTTServerandpresentsasinglepointoffailurewithinthesystem. InkeepingwiththeKISSprincipal(KeepItSimple)allofthemultipleMQTTServertopologiesshownonclusteredMQTTServers.ThemarkettodayoffersmanychoicesofMQTTServers,someofwhichare clusteredinstancesofMQTTServers.Butasabaselinefunctionalityandtokeepthingverysimplethe currentSparkplugspecificationassumesthateachandeveryMQTTServerisastandaloneinstance.One oftheuniquefeaturesofMQTTEngineisthatitseamlesslymanagesthemulti-servertopologies. SparkplugSpecificationVersion1.0 Page48 Figure15-MultipleMQTTServerTopology InthedrawingshowninFigure15above,therearemultipleMQTTServersavailableinthetopology.In thistopology,MQTTEngineconnectstoallavailableMQTTServersintheinfrastructureandprovides detailedmetricsoneachinstance.Thesemetricsinclude: • • • • • • • • NumberofEoNNodesconnectedtoeachMQTTServer LatencycheckinmillisecondsbetweenMQTTEngineandtheMQTTServer. TheMQTTClientIDusedforthisMQTTSession TheOfflineDateTimetimestampofwhentheMQTTServerlastwentoffline. TheOnlineDateTimetimestampofwhentheMQTTServercameonline. ThecurrentrealtimestateoftheconnectiontotheMQTTServer NumberofMQTTmessagespersecondbeingprocessedbythisMQTTServer TheMQTTServerURL SparkplugSpecificationVersion1.0 Page49 Figure16-MQTTClienttagsinIgnition NotethatalloftheIgnitionGatewayinstancesandwellasanyother“LineofBusiness”(LOB)application canestablishMQTTSessionswithallavailableMQTTServers.Inthismanner,theEoNNodesarefreeto determinethebestnetworkpathandMQTTServeravailabilitytoestablishtheirMQTTSessionwith. ThistopologydoesrequirethatEoNNodesusingtheSparkplugspecificationkeepatableofavailable MQTTServersandbeabletoconnecttoanynumberofthembasedonnetworkandserveravailability. UsingthistopologyanyEoNNodecanuseanyoftheavailableTCP/IPnetworksandconnecttoanyone oftheavailableMQTTServers.FromtheIgnitionandLOBapplicationsideofthetopology,theydon’t reallycarewhichoftheNnumberofMQTTServerstheEoNNodesconnectto.TheBirth/Death certificatemanagementautomaticallytakescareofknowwhichEoNNodesareconnectedatanypoint intime,andjustasimportant,whichnodesarenotconnected.MQTTEnginekeepsmetricsonevery EoNNodeconnectingintotheMQTTinfrastructureandincludes: • • • • • • • • CurrentMQTTServerthisEoNNodeisconnectedto. DateTimetimestampofwhenthisEoNwentOfflinelast. DateTimetimestampofwhenthisEoNNodecameOnlinelast. ThenumberofBirthCertificatessentbythisEoNNode. ThenumberofDeathCertificatesseenfromtheMQTTServersonbehalfofthisEoNNode. CurrentOnlinestateofthisEoNNode TotalnumberofactualbytesreceivedfromthisEoNNode. TotalnumberofactualbytessenttothisEoNNode. SparkplugSpecificationVersion1.0 Page50 Figure17-EoNMQTTmetrictagsinIgnition 10.2. PrimaryIgnitionGatewaySTATEinMultipleMQTTServerTopologies ForimplementationswithmultipleMQTTServers,thereisreallyonlyoneadditionalaspectthatneeds tobeunderstoodandmanagedproperly.WhenmultipleMQTTServersareavailablethereisthe possibilityof“stranding”andEoNNodeifthePrimarycommand/controlIgnitionGatewaylosesnetwork connectivitytooneoftheMQTTServers.InthisinstancetheEoNNodewouldstayproperlyconnected totheMQTTServerpublishinginformationnotknowingthatIgnitionwasnotabletoreceivethe messages.WhenusingmultipleMQTTServers,theprimaryIgnitionGatewayinstancemustbe configuredtopublishaSTATEBirthCertificateandallEoNNodesneedtosubscribetothisSTATE message. MQTTEnginehasaconfigurationoptionthatspecifieswhetherthisIgnitionGatewayinstanceisa “Primary”command/controlinstanceornot.IfitisaprimaryinstancetheneverytimeMQTTEngine establishesanewMQTTSessionwithanMQTTServer,theSTATEBirthCertificatedefinedinsection aboveisthefirstmessagethatispublishedafterasuccessfulMQTTSessionisestablished. EoNNodedevicesinaninfrastructurethatprovidesmultipleMQTTServerscanestablishasessionto anyoneoftheMQTTServers.Uponestablishingasession,theEoNNodeshouldissueasubscriptionto theSTATEmessagepublishedbyIgnition.SincetheSTATEmessageispublishedwiththeRETAIN messageflagset,MQTTwillguaranteethatthelastSTATEmessageisalwaysavailable.TheEoNNode SparkplugSpecificationVersion1.0 Page51 shouldexaminethepayloadofthismessagetoensurethatitisavalueof“ONLINE”.Ifthevalueis “OFFLINE”,thisindicatesthethePrimaryIgnitionGatewayhaslostitsMQTTSessiontothisparticular MQTTServer.ThisshouldcausetheEoNNodetoterminateitssessionwiththisMQTTServerandmove tothenextavailableMQTTServerthatisavailable.ThisuseoftheSTATEmessageinthismanner ensuresthatanylossofconnectivitytoanMQTTServertothePrimaryIgnitionGatewaydoesnotresult inEoNNodesbeing“stranded”onanMQTTserverbecauseofnetworkissues.Thefollowingmessage flowdiagramoutlineshowtheSTATEmessageisusedwhenthree(3)MQTTServersareavailableinthe infrastructure: Figure18-IgnitionSTATEflowdiagram 1. WhenanEoNNodeisconfiguredwithmultipleavailableMQTTServersintheinfrastructureit shouldissueasubscriptiontothePrimaryIgnitionGatewaySTATEmessage.TheEoNNodesare freetoestablishanMQTTSessiontoanyoftheavailableserversoveranyavailablenetworkat anytimeandexaminethecurrentSTATEvalue.IftheSTATEmessagepayloadis‘OFFLINE’then theEoNNodeshoulddisconnectandwalktothenextavailableserver. 2. Uponstartup,ifMQTTEngineisconfiguredtobethePrimaryIgnitioninstance,theMQTT SessionwillbeconfiguredtoregisteranIgnitionDEATHCertificatethatindicatesSTATEis ‘OFFLINE’withthemessageRETAINflagsettotrue.ThenanIgnitionBIRTHCertificatewillbe publishedwithaSTATEpayloadof‘ONLINE’. 3. AstheEoNNodewalksitsavailableMQTTServertable,itwillestablishanMQTTSessionwitha serverthathasaSTATEmessagewithapayloadof‘ONLINE’.TheEoNNodecanstayconnected tothisserveraslongasitsMQTTSessionstaysintactanditdoesnotreceiveanIgnitionDEATH Certificate. 4. HavingasubscriptionregisteredtotheMQTTServerontheSTATEtopicwillresultinanychange tothecurrentIgnitionSTATEbeingreceivedimmediately.Inthiscase,anetworkdisruption causestheIgnitionMQTTSessiontoserver#2tobeterminated.ThiswillcausetheMQTT Server,onbehalfofthenowterminatedIgnitionMQTTClienttopublishtheDEATHcertificateto SparkplugSpecificationVersion1.0 Page52 anyonethatiscurrentlysubscribedtoit.UponreceiptoftheIgnitionDEATHCertificatethisEoN NodewillmovetothenextMQTTServerinitstable. 5. TheEoNNodemovedtothenextavailableMQTTServerandsincethecurrentSTATEonthis serveris‘ONLINE’,itcanstayconnected. 6. Inthemeantime,thenetworkdisruptionbetweenIgnitionandMQTTServer#2hasbeen corrected.MQTTEnginehasanewMQTTSessionestablishedtoserver#2withanupdateBirth Certificateof‘ONLINE’.NowMQTTServer#2isreadytoacceptnewEoNNodesessionrequests. SparkplugSpecificationVersion1.0 Page53 11. SparkplugPersistentversusNon-PersistentConnections PersistentconnectionsareintendedtoremainconnectedtotheMQTTinfrastructureatalltimes.They neversendanMQTTDISCONNECTmessageduringnormaloperation.ThisfactletsMQTTEngineprovide therealtimestateofeverypersistentnodeintheinfrastructurewithintheconfiguredMQTTKeepAlive timeperiodusingtheBirth/Deathmechanismsdefinedabove. Butinsomeusecases,suchassendingGPScoordinatesforassettrackingorotherIOTapplicationswith periodicdatafromsensors,MQTTenableddevicesdonotneedtoremainconnectedtotheMQTT infrastructure.Intheseusecases,alltheDeviceneedstodoistoissueanMQTTDISCONNECTcontrol packetpriortogoingofflineinordertoleavetheMQTTinfrastructure“gracefully”.InthiscaseanMQTT DeviceorassociatedDeviceDEATHcertificatewillmostnormallynotbeseen.Systemdesignersjust needtobeawarethatthetagsinIgnitioninthiscasewillrepresent“LastKnownGood”valueswitha timestampofthisdatawherethecurrentstateoftheoftheMQTTDeviceisnotarealtimeindication. Ignitiontagtimestampvaluescanbeusedtodeterminewhenthevaluesfromthisnodewerelast updated. Non-persistentMQTTEnabledDevicesshouldstillregisteraproperDEATHCertificateuponthe establishmentofanMQTTsession.InthismannertheIgnitioncanstillhaveagoodrepresentationof LastKnownGoodprocessvariableversusthefactthattheMQTTsessionwasterminatedpriortothe EoNNodebeingabletocompleteitstransaction. SparkplugSpecificationVersion1.0 Page54 12. GlossaryofTerms ThissectionwillbeprovidedinthenextversionoftheSparkplugspecification. SparkplugSpecificationVersion1.0 Page55 13. ContactInformation ForanyquestionsregardingthisSparkplugspecificationorformoreinformation,pleaseusethe followingdetails: CirruslinkSolutions Website:www.cirrus-link.com Phone:844-924-7787 Email: support@cirrus-link.com SparkplugSpecificationVersion1.0 Page56
