EO Cloud Native Security User Guide Ericsson Orchestrator Operating Instructions 2/1543-AOT 101 5396 Uen AL Copyright © Ericsson AB 2020-2022. All rights reserved. No part of this document may be reproduced in any form without the written permission of the copyright owner. Disclaimer The contents of this document are subject to revision without notice due to continued progress in methodology, design and manufacturing. Ericsson shall have no liability for any error or damage of any kind resulting from the use of this document. Trademark List 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Contents Contents 1 Introduction 1 1.1 Prerequisites 1 2 Product Security Functionality 2 2.1 2.1.1 2.1.2 2.1.3 2.1.4 Role-Based Access Control Access Control for EVNFM Access Control for UDS Access Control for PF Access Control for SO 2 3 6 8 9 2.2 2.2.1 2.2.2 Logging Logging for PF Logging for SO 11 11 12 2.3 Traffic Protection 14 2.4 Network Policy 17 2.5 Multi-tenancy for SO 18 2.6 User Interface for SO 18 3 Security Configuration 19 3.1 3.1.1 3.1.2 3.1.3 3.1.4 3.1.5 3.1.6 3.1.7 Procedures Procedures for EVNFM Procedures for PF Procedures for SO Procedures for UDS Procedures for External LDAP Procedure for Exporting Logs to Remote a Syslog Server Procedure for Configuring Custom Legal Notice 19 19 19 20 21 21 21 22 3.2 Recommended Periodic Operations 23 3.3 Handling of Patches 23 3.4 3.4.1 3.4.2 Security Policies Security Policies for PF Security Policies in SO 23 23 23 3.5 Brute Force Protection 24 4 Default Parameter Values 25 5 Services, Ports, and Protocols 26 5.1 5.1.1 Services, Ports, and Protocols for EVNFM Network Integration Guideline 26 26 5.2 Services, Ports, and Protocols for PF 32 2/1543-AOT 101 5396 Uen AL | 2022-07-25 EO Cloud Native Security User Guide 5.3 Services, Ports, and Protocols for SO 33 5.4 Services, Ports, and Protocols for UDS 33 6 Certificate Management 34 6.1 6.1.1 34 6.3 6.3.1 6.3.2 Certificates in EO Cloud Native Applications Certificates Required for Exporting Logs to Remote Syslog Server Format of the Certificates Required for EO Deployment Server Certificates Client Certificates Verifying Content of Certificates and Key Files Format of the Certificates for Exporting Logs to Remote Syslog Server Procedure to Check for Certificate Expiry Date Procedure to Check Certificate Validity in Google Chrome Procedure to Check Certificate Validity in Mozilla Firefox 6.4 6.4.1 Procedure to Update Certificate in EO Renewing a Certificate 49 49 6.5 Certificate Information for PF 53 6.6 6.6.1 Certificate Information for SO Add SO Subsystem Certificate 54 55 6.7 Certificate Information for UDS 55 6.8 Certificate Information for EO CM 56 6.2 6.2.1 6.2.2 6.2.3 6.2.4 Reference List 38 39 39 41 42 44 46 49 49 57 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Introduction 1 Introduction This document outlines the security processes and mechanisms implemented in the following Ericsson Orchestrator (EO) applications: — Evolved Virtual Network Function Manager (EVNFM) — Policy Framework (PF) — Service Orchestration (SO) — Universal Design Studio (UDS) The Access Management (IdAM) is using OAuth 2.0. 1.1 Prerequisites System administrator rights are required to retrieve logs from identity and access management service of the following EO applications. — Evolved Virtual Network Function Manager (EVNFM) — Policy Framework (PF) — Service Orchestration (SO) — Universal Design Studio (UDS) 2/1543-AOT 101 5396 Uen AL | 2022-07-25 1 EO Cloud Native Security User Guide 2 Product Security Functionality This section describes the following security mechanisms: — Access Control — Logging — Traffic Protection — Network Policy — Multi-tenancy (SO only) — User Interface (SO only) 2.1 Role-Based Access Control With role-based access, a user only has access to the functionality defined in the role assigned by an IdAM administrator. An EO user can be assigned one of the following predefined roles: — MetricsViewer — LogViewer — EO Admin — OSSPortalAdmin — OSSPortalReader The following table identifies the authorized asset actions for each of the available EO roles for both REST and GUI. — Access to REST interfaces is forbidden without the required assigned roles. — Access to the UI is forbidden if the role does not have the correct permissions. Note: 2 These administrator roles are created during installation and are assigned to the default Superusers of each application. 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Product Security Functionality Table 1 EO Assets Actions Role Metric Viewer View metric data and view the graphs. MetricsViewer Log Viewer View system logs LogViewer BUR control Creating backup and restore of the EO system EO Admin OSS Portal Launch GUIs of EO applications, user administration, external LDAP configuration, system logs, and metric data OSSPortalAdmin OSS Portal Launch GUIs of EO applications, system logs, and metric data OSSPortalReader Note: 2.1.1 Authorized Asset Actions and Role Requirement The OSS Portal can be used by SO users with SOProviderAdmin role, and not by users with SOTenantAdmin or SOTenantUser role. Access Control for EVNFM Access to EVNFM is based on user roles. With role-based access, an EVNFM user has access only to the functionality defined in the role assigned by an IdAM administrator. An EVNFM user can be assigned one of the following predefined roles: — EVNFM Super User — EVNFM User — EVNFM Read-Only User — EVNFM UI User Table 2 identifies the authorized asset actions for each of the available EVNFM roles for both REST and GUI. — Access to REST interfaces is forbidden without the required assigned roles. — Only the users with the EVNFM UI User role can perform actions in the GUI. 2/1543-AOT 101 5396 Uen AL | 2022-07-25 3 EO Cloud Native Security User Guide — UI elements associated with an action are not shown without the required assigned roles. Note: The administrator role named EVNFM Super User is created during installation and is assigned to the default EVNFM user. Table 2 Authorized Asset Actions and Role Requirements EVNFM Assets Actions VNF Package Manageme nt Container VNF Life Cycle Manageme nt 4 Role: Role: Role: Role: EVNFM Super User EVNFM User EVNFM Read-Only User EVNFM UI User Query a specified VNF package. Yes Yes Yes No Retrieve details of onboarded VNF packages. Yes Yes Yes No Onboard a VNF package. Yes No No No Delete a VNF package. Yes No No No Query a Yes VNF instance by instance ID. Yes Yes No Query all VNF identifiers. Yes Yes Yes No Instantiate a VNF. Yes Yes No No Terminate a VNF. Yes Yes No No Upgrade a VNF. Yes Yes No No Manual scale of a VNF. Yes Yes No No 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Product Security Functionality EVNFM Assets Actions Role: Role: Role: Role: EVNFM Super User EVNFM User EVNFM Read-Only User EVNFM UI User Add a node Yes to ENM. No No No Delete a node from ENM. Yes No No No Cluster Define Yes Configurati additional on Kubernetes clusters. No No No Log Viewer View system logs. Yes No No No Performan ce Monitoring View and Yes configure performanc e monitor. No No No Back Up and Restore Access Yes back up and restore interface. No No No User Administra tion Create a user. Yes No No No Delete a user. Yes No No No Modify an existing user. Yes No No No View a user. Yes No No No External Create Yes LDAP LDAP user Configurati federation on View LDAP Yes user federation No No No No No No No No No Edit LDAP user federation 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Yes 5 EO Cloud Native Security User Guide EVNFM Assets Actions EVNFM GUI 2.1.2 Role: Role: Role: Role: EVNFM Super User EVNFM User EVNFM Read-Only User EVNFM UI User Delete LDAP user federation Yes No No No Use the GUI. No No No Yes Access Control for UDS Access is based on the roles assigned to the user. A UDS user can be assigned one of the following predefined roles: — UDS Admin — UDS Designer — UDS User The following table identifies the authorized resource actions for each of the available UDS roles. Table 3 Role Based Access Control Resource Role Role Role UDSAdmin UDSDesigner UDSUser Create Yes Yes No Update Yes Yes No Delete Yes Yes No View Yes Yes Yes Create Yes Yes No View Yes Yes Yes Update Yes Yes No Delete Yes Yes No catalog View Yes Yes Yes services Create Yes Yes No View Yes Yes Yes Update Yes Yes No Delete Yes Yes No vendorlicensemodels vendorsoftwareproducts 6 Action 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Product Security Functionality Resource Action Role Role Role UDSAdmin UDSDesigner UDSUser DownloadArtifact Yes Yes No Distribution Yes Yes No Archive Yes Yes No Yes Yes No Yes Yes Yes Yes Yes No DownloadArtifact Yes Yes No Update Yes Yes No Archive Yes Yes No Create Yes Yes No Download Yes Yes No Upload Yes No No Create Yes No No Edit Yes No No View Yes No No Delete Yes No No Onboard Yes No No Update Yes No No Create Yes No No View Yes Yes Yes Edit Yes No No Delete Yes No No Log Viewer View system logs Yes No No Back Up and Restore Access backup and restore interface Yes No No User Administratio n Create a user Yes No No Delete a user Yes No No Modify an existing user Yes No No View a user Yes No No catalogCreate resources(VF& View VFC) Delete policy users artifacts subsystems 2/1543-AOT 101 5396 Uen AL | 2022-07-25 7 EO Cloud Native Security User Guide Resource External LDAP Configuration 2.1.3 Action Role Role Role UDSAdmin UDSDesigner UDSUser Create LDAP user federation Yes No No View LDAP user federation Yes No No Edit LDAP user federation Yes No No Delete LDAP user federation Yes No No Access Control for PF Access is based on the roles assigned to the user. The following table identifies the authorized resource actions for each of the available PF roles. Table 4 Role-Based Access Control Resource Role Role PFAdmin PFUser Create Yes No View Yes Yes Delete Yes No Create Yes No View Yes Yes Delete Yes No Deploy or Undeploy Yes No PDP Group View Yes Yes DMaaP - Events Send or Post event to DMaaP Yes Yes DMaaP - Topics View Yes Yes Publish or Subscribe Yes Yes Create (See note) Yes No Policy Artifact Policy Type Policy Artifact Policy 8 Action 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Product Security Functionality Resource Action Users External LDAP Configuration Note: 2.1.4 Role Role PFAdmin PFUser Create Yes No Edit Yes No View Yes No Delete Yes No Create LDAP user federation Yes No View LDAP user federation Yes No Edit LDAP user federation Yes No Delete LDAP user federation Yes No If a user tries to post an event to a topic that does not exist, the topic is created. Access Control for SO The access is based on roles. The following table identifies the authorized resource actions for each of the available SO roles: Table 5 Role Based Access Control Resourc e Action Artifact s (Service Templa tes, Workflo ws, Jinja Templa tes) Services Role Role Role Role SOProvi derAdm in SODesi gner SOUser Tenant Admin Onboar d Yes Yes No Yes No No View Yes Yes Yes Yes Yes No Delete Yes Yes No Yes No No Create Yes Yes Yes Yes No No 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Role Role SORead Commo Only n Topolog y User 9 EO Cloud Native Security User Guide Resourc e Role Role Role Role SOProvi derAdm in SODesi gner SOUser Tenant Admin View Yes Yes Yes Yes Yes No Delete Yes Yes Yes Yes No No Create Yes No No No No No Edit Yes No No No No No View Yes Yes Yes Yes Yes No Delete Yes No No No No No Create Yes No No No No No Allocate Yes No No No No No View Yes Yes Yes Yes Yes No Delete Yes No No No No No Dealloc ate Yes No No No No No IP Allocate Address Dealloc es ate (IPAM) Yes No No No No No Yes No No No No No UDS Launch Yes Yes No Yes No No Subsyst ems Create Yes No No No No No Edit Yes No No No No No View Yes Yes Yes Yes Yes No Delete Yes No No No No No Create Yes No No No No No View Yes No No No No No Create Yes No No Yes No No Edit Yes No No Yes No No View Yes No No Yes No No Delete Yes No No Yes No No Create LDAP user federati on Yes No No Yes No No View LDAP Yes No No Yes No No Subnet Pools (IPAM) Subnets (IPAM) Tenants Users Externa l LDAP Configu ration 10 Action Role Role SORead Commo Only n Topolog y User 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Product Security Functionality Resourc e Action Role Role Role Role Role Role SOProvi derAdm in SODesi gner SOUser Tenant Admin Edit LDAP user federati on Yes No No Yes No No Delete LDAP user federati on Yes No No Yes No No Workflo w Visualiz ation View No No No No Yes No Commo n Topolog y Datasy nc Create Yes No No No No Yes Commo n Topolog y UI Launch Yes No No No No Yes Service Orders Create Yes Yes Yes Yes No No View Yes Yes Yes Yes Yes No Delete Yes Yes Yes Yes No No View Yes Yes Yes Yes Yes No SORead Commo Only n Topolog y User user federati on Service Catalog 2.2 Logging 2.2.1 Logging for PF Logging records and provides context information for normal operations and it also aids in auditing the system that is used by security operations. It can provide further details, upon request, that help to debug a problem (basic profiling). 2/1543-AOT 101 5396 Uen AL | 2022-07-25 11 EO Cloud Native Security User Guide This information is not sampled as in performance metrics. It is a way to understand how a service request is handled and what happens in a software process. Logging describes: — Definition and categorization of security logs — Identification of a list of events and actions that are required to generate security logs — Impact of security logs — Local storage for security logs Security event logs include recognizing, recording, and storing actions or events. It helps to detect security threats. Sensitive information is not logged in the system or included in the security logs: — passwords — end user identity information — authorization codes — tokens — certificates For troubleshooting and security purposes, the PF logs: — Transaction Logs: • User Transactions • Authorization Security Events — OAuth2 Logs: • 2.2.2 Authentication Security Events - All authentication attempts in PF are logged by default. Logging for SO This is a way of recording events. It contains context information for normal operations and it also aids in audit of the system that is used by security operations. It provides further details on request that help to debug a problem (basic profiling). 12 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Product Security Functionality This information is not sampled as in performance metrics. It is a way to understand how a service request is handled and what happens in a software process. Logging describes: — Definition and categorization of security logs — Identification of a list of events and actions that are required to generate security logs. — Impact of security logs — Local storage for security logs Security event logs include recognizing, recording, and storing actions or events. It helps to detect security threats. Sensitive information is not logged in the system. The following information is not included in the security logs: — passwords — end user identity information — authorization codes — tokens — certificates For troubleshooting and security purposes, the Service Orchestration logs the following information: — Transaction Logs: — User Transactions — Authorization Security Events. — OAuth2 Logs: • Authentication Security Events - All authentication attempts in Service Orchestration are logged by default. See Procedures for SO on page 20 for instructions on how to access logs. 2/1543-AOT 101 5396 Uen AL | 2022-07-25 13 EO Cloud Native Security User Guide 2.3 Traffic Protection The following protocols are used for traffic protection in EO applications: Table 6 Application EVNFM Protocols — HTTPS — SFTP Usage — Secure HTTP prevents tampering of communications between the system and the browser. It also prevents passive listening-in on communication between service and the users. — Secure FTP ensures that data is securely transferred using a private and safe data stream. — Self-signed certificates can be generated and used in a test environment. For security reasons, they are not recommended for a production environment. Certificates must be obtained from a trusted Certificate Authority. PF 14 HTTPS — External client communication for all PF interfaces is secured through HTTPS. It uses HTTP IETF RFC7230 over a Transport Layer Security V1.2 connection (TLS) 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Product Security Functionality Application Protocols Usage IETF RFC5246). Unsecured external client communication is not possible. — Self-signed certificates can be generated and used in a test environment. For security reasons, they are not recommended in a production environment. Certificates must be obtained from a trusted certificate authority. SO HTTPS — External client communication for all SO interfaces is secured through HTTPS. It uses HTTP IETF RFC7230 over a Transport Layer Security V1.2 connection (TLS) IETF RFC5246). Unsecured external client communication is not possible. — Self-signed certificates can be generated and used in a test environment. For security reasons, they are not recommended in a production environment. Certificates must be obtained from a trusted certificate authority. 2/1543-AOT 101 5396 Uen AL | 2022-07-25 15 EO Cloud Native Security User Guide Application Protocols UDS HTTPS Usage — SO is allowed to communicate with southbound systems (EO CM or ENM) over HTTPS, though it cannot verify the certificates. This is a low security risk as the product is running in a controlled environment, behind a firewall. — External client communication for all UDS interfaces is secured through HTTPS. It uses HTTP IETF RFC7230 over a Transport Layer Security V1.2 connection (TLS) IETF RFC5246). Unsecured external client communication is not possible. — Self-signed certificates can be generated and used in a test environment. For security reasons, they are not recommended in a production environment. Certificates must be obtained from a trusted certificate authority. — UDS is allowed to communicate with southbound systems (EO CM /ENM) over HTTPS, though it cannot verify the certificates. This is a 16 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Product Security Functionality Application 2.4 Protocols Usage low security risk as the product is running in a controlled environment, behind a firewall. Network Policy Network Policy is a specification of how groups of pods are allowed to communicate with each other and other network endpoints. The network policy is a set of rules for the connectivity access of microservices. It defines how policies are enforced and lays out the basic architecture of the environment. It governs data access, passwords, and encryption for individuals or groups of individuals. The purpose is to ban malicious users and have control over potential risk users. The network policies within the EO applications determine what information and services are available to which users. In addition, the network policy also determines the hierarchy of access permissions. The underlying infrastructure decodes and secures the network of an application based on the policies defined. Stages 1. Deny all internal (ingress) traffic policy is enabled: — A default isolation policy for the name space that selects all pods but does not allow any ingress traffic to those pods. — Assign traffic to an allowlist as defined in the Network Policy. — Temporary isolation of traffic from a Service to, or, from other Pods. 2. Set Traffic manually for each microservice: — For SO, there is a special case scenario for the service checker. It communicates with the external clients, internal clients, and also with the microservices. — For different microservices, the internal traffic is responsible for the communication. — For the API-Gateway microservice, traffic is from the external clients. 2/1543-AOT 101 5396 Uen AL | 2022-07-25 17 EO Cloud Native Security User Guide 2.5 Multi-tenancy for SO Multi-tenancy allows isolation of data. Only users belonging to a tenant have access to the data of the tenant. The following data belongs to a tenant: — Services — Service resources, for example, NetworkService and NetworkFunction — Actions — Execution Plans 2.6 User Interface for SO Service Orchestration has implemented solutions to prevent the following attacks: — Clickjacking - It is a malicious technique of tricking a user to click a disguised link, which has the potential to steal confidential information or allow others to take control of the computer. — XSS attacks - X-XSS Protection response header stops pages from loading, when they detect reflected XSS attacks. Modern browsers implement a strong Content-Security-Policy (CSP) that disables the use of inline JS ('unsafe-inline'). Protection is provided for users of older versions of web browsers that do not support CSP. — Multipurpose Internet Mail Extension (MIME) type Sniffing: The X-ContentType Options response HTTP header is set to 'nosniff'. The X-Content-Type Options response HTTP header is a marker used by the server to indicate that the MIME types advertised in the Content-Type headers must not be changed and be followed. This allows to opt-out of MIME type sniffing. — Strict Transport Security: The HTTP Strict-Transport-Security response header (HSTS) lets a website tell browsers that it must only be accessed through HTTPS, instead of HTTP. It also prevents HTTPS click through prompts on browsers. 18 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Security Configuration 3 Security Configuration This section describes how to operate the security functionality of the EO applications. 3.1 Procedures This section describes the procedures to collect information about the daily operations that the user must perform. 3.1.1 Procedures for EVNFM Not applicable. 3.1.2 Procedures for PF Viewing Identity and Access Management Security Logs Security logs are enabled in the identity and access management service of PF. Security logs include login events and account events. Login events occur when a user accesses PF: — Login - A user has logged on to a tenant. — Login Error - An unsuccessful login attempt was made. — Logout - A user has logged out of a tenant. — Update Password - A new user has updated the password at first login. — Update Password Error - An unsuccessful attempt to update the password was made. — Code to Token - An application or client has exchanged a code for a token. — Refresh Token - An application or client has refreshed a token. Account events occur when user management activities take place. — Create - A user has been created in a tenant. — Delete - A user has been deleted from a tenant. 2/1543-AOT 101 5396 Uen AL | 2022-07-25 19 EO Cloud Native Security User Guide Location of Security Logs and Secure Transfer of Logs To retrieve the logs, the kubectl CLI is used and logs are retrieved from the Policy Framework Identity and Access Management pod: ssh -i<path/to/key/file> <username>@<ip> kubectl -n <pf-namespace> logs eric-sec -access-mgmt-0' >> IAM_Logs.txt &8594; '. Note: 3.1.3 → Only the system administrator can retrieve logs from identity and access management service of PF. Procedures for SO Viewing Identity and Access Management Security logs Security logs are enabled in the identity and access management service of SO. Security logs include login events and account events. Login events occur when a user accesses SO: — Login - A user has logged on to a tenant. — Login Error - An unsuccessful login attempt is made. — Logout - A user has logged out of a tenant. — Update Password - A new user has updated the password at first login. — Update Password Error - An unsuccessful attempt to update the password is made. — Code to Token - An application or client has exchanged a code for a token. — Refresh Token - An application or client has refreshed a token. Account events occur when user management activities take place: — Create - A user is created in a tenant. — Delete - A user is deleted from a tenant. Location of Security Logs and Secure Transfer of Logs To retrieve the logs, the kubectl CLI is used and logs are retrieved from the Service Orchestration Identity and Access Management pod: ssh -i<path/to/key/file> <username>@<ip> kubectl -n so logs eric-sec-access-mgmt-0' >> IAM_Logs.txt '. Note: 20 Only the system administrator can retrieve logs from identity and access management service of SO. 2/1543-AOT 101 5396 Uen AL | 2022-07-25 → Security Configuration 3.1.4 Procedures for UDS Not applicable. 3.1.5 Procedures for External LDAP To configure a SSL connection between IAM and Lightweight Directory Access Protocol (LDAP), a root CA certificate for the external LDAP must be injected into EO. Steps 1. Copy the contents of LDAP tls certificate file (CA root certificate file with extension .crt used for LDAP SSL configuration) into the intermediateca.crt file used when EO was installed (append certificates to existing ones, keep all the previous content intact). The file is used in the next steps to update the iam-cacert-secret secret. 2. Do certificate update procedure, see Renewing a Certificate. Perform all the steps described for a client certificate. Note: 3.1.6 All instances of the keycloak microservice (eric-sec-access-mgmt) require restarting to load in the new certificate. Procedure for Exporting Logs to Remote a Syslog Server To export logs to remote Syslog server, one trusted certificate named syslogtrustedcert.crt (CA used to sign the Syslog certificates), and a certificate or key pair named log-transformer-syslog.crt (client certificate for Syslog) and log-transformer-syslog.key (key for client certificate for Syslog) must be injected into EO. Steps 1. Create trusted certificate syslog-trustedcert.crt, see section 6.1.2 Certificate required for exporting logs to remote Syslog server and section 6.2.4 Format of Certificate for exporting logs to remote Syslog server. 2. Create certificate or key pair log-transformer-syslog.crt and logtransformer-syslog.key. see section 6.1.2 Certificate required for exporting logs to remote Syslog server and section 6.2.4 Format of Certificate for exporting logs to remote Syslog server. 3. Add the above files into the certificates folder in your workdir directory before starting the EO installation or upgrade, see section 4.7.1 EO Install Instructions for Management of certificate to Export Logs to Remote Syslog Server and section 4.7.1 Upgrade Instructions for Management of certificate to Export Logs to Remote Syslog Server. 2/1543-AOT 101 5396 Uen AL | 2022-07-25 21 EO Cloud Native Security User Guide 4. Configure Syslog Parameters to enable support sending the logs to remote host, see section 7.1 EO Install Instructions for Required Values for the necessary Site Values File configuration and section 9.1 Upgrade Instructions for Required Values for the necessary Site Values File configuration. The syslog-trustedcert.crt file is used to create the secret eric-logtransformer-trusted-external-secret. The log-transformer-syslog.crt and log-transformer-syslog.key files are used to create the secret eric-log-transformer-asymmetric-secret. Both secrets are required to export logs to a remote Syslog server. 3.1.7 Procedure for Configuring Custom Legal Notice This section provides information on configuring custom Legal Notice for EO. IdAM is already populated with a pre-defined Legal Notice, which is accessible by clicking the legal notice link on the login page. Instead of a pre-defined legal notice, a custom legal notice can be set at deployment time. Steps 1. For displaying custom Legal Notice on IdAM Login Page, in the generated site_values_<EO_VERSION>.yaml file (for details on this file, see EO Cloud Native Installation Instructions), add the following configuration. There are already other properties set under eric-cloud-native-base.eric-secaccess-mgmt, add these next to them. For description of Custom Legal Notice parameter, see EO Cloud Native Installation Instructions. eric-cloud-native-base: eric-sec-access-mgmt: preLoginMessage: messageString: '' 2. Set the custom Legal Notice as a value for the previous configuration as follows: eric-cloud-native-base: eric-sec-access-mgmt: preLoginMessage: messageString: 'give here custom legal-notice' For displaying multi-line custom Legal Notice, YAML syntax can be used as follows. 22 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Security Configuration eric-cloud-native-base: eric-sec-access-mgmt: preLoginMessage: messageString: | give custom legal notice here some more legal notice here . Note: 3.2 If no value is set for the previous configuration, the pre-defined Legal Notice from IdAM is displayed on the login page. Recommended Periodic Operations Not applicable. 3.3 Handling of Patches No specific patches are delivered. Any updates are pushed through new versions of the application releases and require a new install or an upgrade. 3.4 Security Policies 3.4.1 Security Policies for PF The following table shows the default security-related policies and their configurations: Table 7 3.4.2 Security Policies Policy Configuration Default Password must not be the same as the username. Yes Length of password must be at least 8 characters. Yes Password must contain the following elements: Yes — At least 1 lower case — At least 1 upper case — At least 1 numeric — At least 1 special character Security Policies in SO The following table shows the default security-related policies and their configurations: 2/1543-AOT 101 5396 Uen AL | 2022-07-25 23 EO Cloud Native Security User Guide Table 8 Security Policies Policy Configuration Default Password must not be the same as the Yes username. Length of password must be at least 8 characters. Yes Password must contain the following elements: Yes — At least 1 lower case — At least 1 upper case — At least 1 numeric — At least 1 special character 3.5 Brute Force Protection All EO applications are configured to be protected against brute force attacks. This prevents an attacker from continually guessing a user password and gaining access to the system. If 30 unsuccessful log-on attempts are made over a 12hour period for the same user, that user is locked out for up to 15 minutes. If two unsuccessful attempts are made within a second, then the user is locked out for up to 15 minutes. 24 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Default Parameter Values 4 Default Parameter Values The default values for the security parameters are listed in the following table: Table 9 Default Parameter Values Parameter Default Value Not applicable No default values 2/1543-AOT 101 5396 Uen AL | 2022-07-25 25 EO Cloud Native Security User Guide 5 Services, Ports, and Protocols 5.1 Services, Ports, and Protocols for EVNFM The services, ports, and protocols that are used by EVNFM are listed in the following table: Table 10 5.1.1 Services, Ports, and Protocols Service or Interface Name Protocol IP Address Type Port Transport Protocol IP Version Web Interface HTTPS OAM IP 443 TCP IPv4 Docker Registry HTTPS OAM IP 443 TCP IPv4 IAM HTTPS AOM IP 443 TCP IPv4 SFTP Server Access SFTP STATIC 22 FTP IPv4 Network Integration Guideline VM VNFM The communication flows representation in terms of "from/to" refers to the expected initial communication setup (only the documented unidirectional communication initiation flows have to be allowed on relevant firewalls). Unless otherwise specified, the source port number is expected to fall into the ephemeral port range (OS dependent, usually in the 1024–65535 port range). Table 11 26 VM VNFM to ENM Services Application Transport Protocol Protocol Source Node DST Port Number Destination Comment Node IIOP/ GIOP TCP VM VNFM (Worker Node) 9951 ENM (LVS Router) to VM VNFM - IIOP/ GIOP TCP VM VNFM (Worker Node) 9954 ENM (visinaami ngnb) - 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Services, Ports, and Protocols Application Transport Protocol Protocol Source Node DST Port Number Destination Comment Node SSH/SFTP TCP VM VNFM (Worker Node) 22 ENM (scp-1scripting) - NTP TCP VM VNFM (Worker Node) 123 ENM (scp-1scripting) - DST Port Number Destination Comment Node Table 12 ENM Services to VM VNFM Application Transport Protocol Protocol Source Note IIOP/ GIOP TCP ENM(visina 9002 mingnb, nbalarmirp ) VM VNFM(exte rnal ingress lb) HTTP TCP ENM(svc 80 cluster node,Ivsro uter,haprox y) VM VNFM(exte rnal ingress lb) HTTPs TCP ENM(svc 443 cluster node,Ivsro uter,haprox y) VM VNFM(exte rnal ingress lb) HTTP TCP ENM(svc 8080 cluster node,Ivsro uter,haprox y) VM VNFM(exte rnal ingress lb) IIOP/ GIOP TCP ENM(visina 9954 mingnb, nbalarmirp ) VM VNFM(exte rnal ingress lb) Table 13 VM VNFM to CEE/OpenStack Application Transport Protocol Protocol Source Node DST Port Number Destination Comment Node HTTPs VM VNFM (Worker Node) 5000 ECEE TCP 2/1543-AOT 101 5396 Uen AL | 2022-07-25 VM VNFM contacts the Cloud Execution 27 EO Cloud Native Security User Guide Application Transport Protocol Protocol Source Node DST Port Number Destination Comment Node Environme nt (CEE) Identity Service REST API port to authenticat e with CEE. Default port and exact port numbers must be checked for the specific CEE deploymen t. HTTPs TCP VM VNFM (Worker Node) 8004 ECEE VM VNFM contacts the Atlas (OpenStac k Heat) API port for Cloud resource orchestrati on, such as instantiatio n, scaling, and terminatio n of VNFs. Default port and exact port numbers must be checked for the specific CEE deploymen t. HTTPs 28 TCP VM VNFM (Worker Node) 8774 ECEE The workflows contact the CEE NOVA Compute 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Services, Ports, and Protocols Application Transport Protocol Protocol Source Node DST Port Number Destination Comment Node Service REST API port to create virtual machines on CEE. Default port and exact port numbers must be checked for the specific CEE deploymen t. HTTPs TCP VM VNFM (Worker Node) 9292 ECEE The workflows contact the CEE Glance Service REST API port for querying of VM image metadata and for retrieving the actual images on CEE. Default port and exact port numbers must be checked for the specific CEE deploymen t. HTTPs TCP 2/1543-AOT 101 5396 Uen AL | 2022-07-25 VM VNFM (Worker Node) 9696 ECEE The workflows contact the CEE Neutron Service 29 EO Cloud Native Security User Guide Application Transport Protocol Protocol Source Node DST Port Number Destination Comment Node REST API port to install and configure the networking service on CEE. Default port and exact port numbers must be checked for the specific CEE deploymen t. HTTPs TCP VM VNFM (Worker Node) 8776 ECEE The workflows contact the CEE Cinder Service REST API port to install and configure the networking service on CEE to provide block storage devices to the guest instances on CEE. Default port and exact port numbers must be checked for the specific CEE deploymen t. 30 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Services, Ports, and Protocols Application Transport Protocol Protocol Source Node DST Port Number Destination Comment Node HTTPs TCP VM VNFM (Worker Node) 443 ECEE HTTPs TCP VM VNFM (Worker Node) 8082 ECEE The workflows contact the CEE Contrail Service REST API port to install and configure the Contrail service on CEE to provide block storage devices to the guest instances on CEE. Default port and exact port numbers must be checked for the specific CEE deploymen t. Table 14 VM VNFM to VNFs Application Transport Protocol Protocol Source Node DST Port Number Destination Comment Node SSH VM VNFM (Worker Node) 22 VNFs TCP SSH default port For VNFspecific ports, see the 2/1543-AOT 101 5396 Uen AL | 2022-07-25 31 EO Cloud Native Security User Guide Application Transport Protocol Protocol Table 15 Source Node DST Port Number Destination Comment Node respective VNF section. 3PP Orchestrator to VM VNFM Application Protocol Transport Protocol Source Node DST Port Number Destination Node HTTPs TCP 3PP Orchestrator 443 VM VNFM(extern al ingress lb) Table 16 VM VNFM to NFVO Example of NFVO: EO CM Note: 5.2 This table contains default DST Port Number of NFVO 443, 8080 and 80. Contact NFVOadministrator to obtain the right port. Application Protocol Transport Protocol Source Node DST Port Number Destination Node HTTP TCP VM VNFM(extern al ingress lb) 80 NFVO HTTP TCP VM VNFM(extern al ingress lb) 8080 NFVO HTTPs TCP VM VNFM(extern al ingress lb) 443 NFVO Services, Ports, and Protocols for PF The services, ports, and protocols that are used by the product are listed in the following table. Table 17 32 Services, Ports, and Protocols Service or Interface Name Protocol IP Address Type Port Transport Protocol IP Version Web Interface HTTPS OAM IP 443 TCP IPv4 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Services, Ports, and Protocols 5.3 Services, Ports, and Protocols for SO The services, ports, and protocols that are used by the product are listed in the following table: Table 18 5.4 Services, Ports, and Protocols Service or Interface Name Protocol IP Address Type Port Transport Protocol IP Version Web Interface HTTPS OAM IP 443 TCP IPv4/IPv6 Backup and Restore SFTP OAM IP 22 TCP IPv4/IPv6 Services, Ports, and Protocols for UDS The services, ports, and protocols that are used by the product are listed in the table. Table 19 Services, Ports, and Protocols Used by UDS Service or Interface Name Protocol IP Address Type Port Transport Protocol IP Version Web Interface HTTPS OAM IP 443 TCP IPv4 IAM HTTPS OAM IP 443 TCP IPv4 Back up and Restore SFTP OAM IP 22 TCP IPv4 2/1543-AOT 101 5396 Uen AL | 2022-07-25 33 EO Cloud Native Security User Guide 6 Certificate Management This section describes the processes involved in maintaining the ssl/tls certificates that are required to run the EO Cloud Native applications securely. Note: 6.1 The ssl/tls certificates signed by the private or public certificate signing authorities are supported in the EO Cloud Native applications. Selfsigned certificates are not supported in the EO Cloud Native Applications. Certificates in EO Cloud Native Applications The following table gives an overview of all the certificates that are used in the deployment and execution of the EO Cloud Native Applications. All the secrets mentioned in the following table are required during deployment. Table 20 List of Certificates in EO Cloud Native Applications Applications Secrets EVNFM, SO, UDS, PF, GAS iam-tls-secret Certificate Information Client/Server — TLS certificate to be Server injected into the ingress configuration to enable TLS termination at the application ingress. — IAM ingress uses this secret to configure the TLS in the ingress controller. EVNFM, SO, UDS, PF, GAS iam-cacertsecret — CA certificate of IAM server. Client — CA Certificates of any DomainManagers or Network Controllers to be used with SO. EVNFM 34 vnfm-tls-secret — api-gateway TLS certificate Server 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Certificate Management Applications Secrets EVNFM registry-tlssecret Certificate Information Client/Server — TLS certificate to be injected into the ingress configuration to enable TLS termination at the application ingress. — api-gateway TLS certificate Server — TLS certificate to be injected into the ingress configuration to enable TLS termination at the application ingress. EVNFM registry-tlssecret Note: The following action is required only when using unrecognized signing authority for creating certificates. Client To support different container handling clients, the procedure for setting up the certificates vary. Perform the following procedure, which matches your use case: — Docker: If the registry CA certificate is not present in the worker node, then the complete certificate chain must be added in all the worker node to the /etc/ docker/certs.d/ <ingress host of 2/1543-AOT 101 5396 Uen AL | 2022-07-25 35 EO Cloud Native Security User Guide Applications Secrets Certificate Information Client/Server container registry>/ca.crt directory. This is to enable the worker node to pull the docker images from the registry microservice. This step is redundant if the registry server signing certificate is present in the worker node. Required when certificate created with unrecognized signing authority. — Containerd: If the registry CA certificate is not present in the worker node, add the complete certificate chain to all the worker nodes. This can be achieved by executing the following steps: 1.Add the complete certificate chain to the following location on the worker node: /etc/pki /trust/ anchors/ <complete_chai n_file_name>.c rt 2.Execute sudo update-cacertificates on the worker node to refresh the SSL Certificates. 36 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Certificate Management Applications Secrets Certificate Information 3.Restart the containerd process on the worker node by executing sudo Client/Server systemctl restart containerd. EVNFM helm-registrytls-secret Note: Only required if Server exposing helm registry through ingress. — This is the TLS certificate and key of helm registry microservice. The EVNFM Workflow service must mount this certificate and add it to the /etc/ssl if the signing certificate of the registry server certificate is not present in the workflow service pod. — EVNFM currently only supports externally signed certificate for helmregistry certificate. SO so-tls-secret — SO GUI/NBI Server — TLS certificate to be injected into the ingress configuration to enable TLS termination at the application ingress. UDS 2/1543-AOT 101 5396 Uen AL | 2022-07-25 uds-tls-secret — TLS certificate to be Server injected into the ingress configuration to enable TLS termination at the application ingress. 37 EO Cloud Native Security User Guide Applications Secrets PF pf-tls-secret Certificate Information — Policy Framework TLS Certificate Client/Server Server — TLS certificate to be injected into the ingress configuration to enable TLS termination at the application ingress. EVNFM GAS eric-eo-evnfmnfvo — NFVO TLS certificate Client ga-tls-secret — OSS Portal TLS certificate — For EVNFM, the certificate is added to the Keystore of the EVNFM onboarding service when the secret is created. Server — TLS certificate to be injected into the ingress configuration to enable TLS termination at the application ingress. EO CM 6.1.1 eric-eo-cm-nbitls-cert — EO CM GUI/NBI Server — TLS certificate to be injected into the ingress configuration to enable TLS termination at the application ingress. Certificates Required for Exporting Logs to Remote Syslog Server The syslog-trustedcert.crt file is used to create the secret eric-logtransformer-trusted-external-secret. The log-transformer-syslog.crt and log-transformer-syslog.key files are used to create the secret eric-logtransformer-asymmetric-secret. Both secrets are required to export logs to a remote Syslog server. 38 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Certificate Management Table 21 6.2 List of Certificates in Exporting Logs to Remote Syslog Server File File Information Client/Server log-transformersyslog.crt A certificate which must be paired to logtransformer-syslog.key and generated using a trusted key (client certificate for Syslog). Client log-transformersyslog.key A key which must be Client paired to the logtransformer-syslog.crt and generated using a trusted key (key for client certificate for Syslog). syslogtrustedcert.crt A trusted certificate that Client must also appear on the syslog server (CA used to sign the Syslog certificates). Format of the Certificates Required for EO Deployment This section describes the format of the certificates that are required for an EO deployment. 6.2.1 Server Certificates Server certificates are used with ingresses to allow HTTPS access to a given application. 6.2.1.1 Server Certificates Configuration Requirements When generating server certificates, subjectAltName field must be set to include DNS: <FQDN> entry. This can be verified using the following command: openssl x509 -in <Servert Certificate File Name> -text -noout The output of the previous command must contain the following lines: 2/1543-AOT 101 5396 Uen AL | 2022-07-25 39 EO Cloud Native Security User Guide X509v3 Subject Alternative Name: DNS:<FQDN> 6.2.1.2 Bundled Format for Server Certifications A bundle contains a list of certificates, where it must contain at least two certificates. Generally, it can be a root CA, intermediate CA, or a full chain, where the full chain includes a server certificate, intermediate CA certificate, and the root CA certificate. Intermediate CA is a sub CA that signs your server certificate behalf of the root CA. In a situation where the intermediate CA or root CA can be unknown, the full chain is required in the bundle. The certificates issued by a trusted CA can have different extensions such as .cer,.pem, .crt, or .cert but they must be in the same PEM encoding. Only certificates in the base64 encoding format are supported. Note: The commands in following examples must be run from a client where you have openssl command available. It is assumed that the respective certificates and keyfiles are available in the current working directory of the particular environment. Run the following command to create a server certificate bundle. $ cat <Server Certificate> <Signing CA Certificate(s)> > <FQDN>. → crt #<Server Certificate> : A server certificate is used to authent icate the server’s identity to the Client. #<Signing CA Certificate(s)>: The signing certificate(s) is the certificate(s) of the CA(s) who signed / issued the server cert ificate to us. It could be a intermediate CA or a root CA or eve n the full chain. #<FQDN>.crt : It is the bundle file (with the Fully Qualified Do main Name) which contains both the certificate. Example output: -----BEGIN CERTIFICATE----MIIDozCCAougAwIBAgIUFdhgJ0qCUc1nVCwa0lOOYtxikxwwCQYDVQQHDAJhYjEL EwJJcjELMAkGA1UECAwCYWIxCzAJBgNVBAcMAmFiMQswCQYDVQQKDAJhYjELMAkG ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ 1GV4ectNVSkjCYau29c9C2pPwsJ2F8AHXGAv7xkD9+1sgGkIzKWEUV2C1eIJ657I jSfV4CMDRha7rD1AO7xHTpfrr2FRJbc= -----END CERTIFICATE---------BEGIN CERTIFICATE----MIIDqzCCApOgAwIBAgIUTgSUAE4QPuvniPjmuNfbexL51MkwDVQQHDAJzYTELAkG 40 2/1543-AOT 101 5396 Uen AL | 2022-07-25 → → → → → Certificate Management A1UEBhMCSXIxCzAJBgNVBAgMAnNhMQswCQYDVQQHDAJzYTELMAkGA1UECgwCc2Ex ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ LwsdvI7tW+qn3ffiWiarLt/Ft/95zJSBgeiQ+OqOBG2zSqChdHvHsnQcJ1f5npvz x1SwoZCZVHjhZT14ygp3Tvs3yG7Y3j9/B+otNJMuiQ== -----END CERTIFICATE----- 6.2.1.3 Key Format of Server Certificate Keys A key file must be in PEM format. To open a key file, use any standard text editor and it must be displayed as shown in the following example. In documentation, the key file is referred as <FQDN>.key. -----BEGIN RSA PRIVATE KEY----MIIEpAIBAAKCAQEAz25EAs03nVAWE3m8OjY7Hoclw2jsA+OAGD9cpSTeudapOV2v bI6tGaBmot8HWC1Tws1nE6RIZYtuoihESTD1ZEgIxPH4iyOjrr/2k/hNFlldMw8z ................................................................ ................................................................ ................................................................ ................................................................ I2bfVt7UQAb8JTLxyVw1MIAwT7nCK36vf5/UsjMqxlRlW8+eplK6XAXeoomb8cgo vyXutPWvpNL2/SqkxPbI6f1Irp/Nxc2bz23gbalysgUDNt7m5UKi9A== -----END RSA PRIVATE KEY----- 6.2.2 Client Certificates Client certificates are used with different micro-services in EO to verify SSL communication with another service or subsystem. 6.2.2.1 Bundled Format for Client Certificates A Client certificate is a type of certificate that is used to validate or trust the issuer of another server certificate. A client certificate must be in the PEM encoding format like rest of the certificates used in the system. Only certificates in the base64 encoding format are supported. An example for a secret that mounts a client certificate in EO is iam-cacert-secret or eric-eo-evnfm-nfvo secret. To create a Kubernetes secret for a client certificate, a key file is not required. The client certificates are only used by microservices to validate a server certificate when communicating with a particular HTTPS endpoint. The certificate content of a secret containing a client certificate is mounted as a file to different 2/1543-AOT 101 5396 Uen AL | 2022-07-25 41 EO Cloud Native Security User Guide microservices in the system. The client certificate includes the certificate of the signing CA for a given server certificate. There can be one or more CA certificates present within a single client certificate file. Following is an example output of a client certificate. In this example output, only a single certificate is available in the client certificate file. -----BEGIN CERTIFICATE----MIIDozCCAougAwIBAgIUFdhgJ0qCUc1nVCwa0lOOYtxikxwwCQYDVQQHDAJhYjEL EwJJcjELMAkGA1UECAwCYWIxCzAJBgNVBAcMAmFiMQswCQYDVQQKDAJhYjELMAkG ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ 1GV4ectNVSkjCYau29c9C2pPwsJ2F8AHXGAv7xkD9+1sgGkIzKWEUV2C1eIJ657I jSfV4CMDRha7rD1AO7xHTpfrr2FRJbc= -----END CERTIFICATE----- 6.2.2.2 Key Format of Client Certificate Keys No key is required with the client certificate. 6.2.3 Verifying Content of Certificates and Key Files This section describes the commands to verify the content of an SSL certificate or a key file. 1. Print the content of a base64 encoded certificate in a human readable format: $ openssl x509 -in <FQDN>.crt -noout -text Example Certificate: Data: Version: 1 (0x0) Serial Number: 37:2d:7c:d8:6e:61:46:6a:c1:58:e1:2a:d0:a6:6d:b6:5e:b1:39:9a Signature Algorithm: sha256WithRSAEncryption Issuer: C = IE, ST = Westmeath, L = Athlone, O = Internal Intermedia → te CA, OU = Security, CN = internalintermediateca.com, emailAddress = admin@ → internalintermediateca.com Validity Not Before: Jul 16 12:05:28 2020 GMT Not After : Jul 11 12:05:28 2021 GMT Subject: C = IE, ST = Westmeath, L = Athlone, O = My App, OU = Web, → CN = myapp.com, emailAddress = admin@myapp.com Subject Public Key Info: Public Key Algorithm: rsaEncryption 42 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Certificate Management RSA Public-Key: (2048 bit) Modulus: 00:cf:22:f8:4e:e2:47:f5:2f:e2:47:9e:a8:94:55: ............................................: ............................................: ............................................: ............................................: Exponent: 65537 (0x10001) Signature Algorithm: sha256WithRSAEncryption c3:dc:6c:b7:c8:ce:74:6f:46:d4:63:7c:92:d0:78:79:a4:37: .....................................................: .....................................................: .....................................................: .....................................................: 2. Verify if a key is in PEM encoding: $ openssl rsa -in <FQDN>.key -noout -text Example RSA Private-Key: (2048 bit, 2 primes) modulus: 00:cf:6e:44:02:cd:37:9d:50:16:13:79:bc:3a:36: ............................................: ............................................: ............................................: ............................................: publicExponent: 65537 (0x10001) privateExponent: 4b:38:bb:65:20:b2:68:53:e0:8d:93:79:d4:ca:51: ............................................: ............................................: ............................................: ............................................: ............................................: prime1: 00:f5:dd:64:d4:4c:98:8d:80:ba:22:fd:87:d2:80: ............................................: ............................................: ............................................: ............................................: ....................... prime2: 00:d7:fb:47:12:fe:45:83:35:8d:84:bb:e9:9c:7e: ............................................: ............................................: ............................................: ............................................: ....................... exponent1: 00:be:ab:1a:ee:a9:18:05:64:b6:f5:3b:b8:81:3e: ............................................: ............................................: ............................................: ............................................: ....................... exponent2: 29:88:b5:fb:4e:10:9a:11:e3:5c:22:32:e3:98:61: ............................................: ............................................: ............................................: ............................................: ....................... coefficient: 00:a8:48:67:96:e7:96:59:70:5b:6c:25:af:be:cd: ............................................: ............................................: ............................................: ............................................: ....................... 2/1543-AOT 101 5396 Uen AL | 2022-07-25 43 EO Cloud Native Security User Guide All the above certificates must be in the X509 format and PEM encoded. 6.2.3.1 Error Messages in Verification Steps If the certificates or key are not PEM encoded or if the file contains any garbage characters, then the commands described in the Verifying Content of Certificates and Key Files on page 42 section throw an error. Following are the example outputs of such error messages. Prerequisites: — The System Administrator must have access to the kubernetes cluster and the environment on which openssl is installed. 1. Example verification output for a certificate which is not PEM encoded but in DER format. unable to load certificate 540409864:error:0909006C:PEM routines:get_name:no start line:crypto/pem/pem_ → lib.c:745:Expecting: TRUSTED CERTIFICATE 2. Example verification output for a certificate which has garbage characters. unable to load certificate 540409864:error:09091064:PEM routines:PEM_read_bio_ex:bad base64 decode:cryp → to/pem/pem_lib.c:929: If any certificate fails the verification steps, contact the certificate authority. 6.2.4 Format of the Certificates for Exporting Logs to Remote Syslog Server This section describes the format of the certificates and the keys that are required for exporting logs to the remote Syslog server. 1. Example of syslog-trustedcert.crt The syslog-trustedcert.crt file is used to create the secret eric-logtransformer-trusted-external-secret. -----BEGIN CERTIFICATE----MIIFVzCCAz+gAwIBAgIJAPM3iS5eHkaEMA0GCSqGSIb3DQEBCwUAMEIxCzAJB gNV BAYTAlhYMRUwEwYDVQQHDAxEZWZhdWx0IENpdHkxHDAaBgNVBAoME0RlZmF1b HQg ............................................................. ... ............................................................. ... 44 2/1543-AOT 101 5396 Uen AL | 2022-07-25 → → → → Certificate Management ............................................................. ... ............................................................. ... ............................................................. ... agH/wGsiQChp0sr+IkLhQkDJQhWXa0bplk7nFlATyEMpyxT4TFzjmtTCY0HK5 9Kc 70d3z1OiO+UrvAgrZ1tP5ya9Xzmz4cHPPXHBq/4bnupT+aVSyekEXjBe3bb3A VHa Yl6YFVo+Hf81h/N+PmMs7XIlVtBscRnx+gOD -----END CERTIFICATE----- → → → → → 2. Example of log-transformer-syslog.crt The log-transformer-syslog.crt alongside log-transformersyslog.key is used to create the secret eric-log-transformerasymmetric-secret. -----BEGIN CERTIFICATE----MIID3TCCAcUCCQCb2ktCrQoUOjANBgkqhkiG9w0BAQsFADBCMQswCQYDVQQGE wJY WDEVMBMGA1UEBwwMRGVmYXVsdCBDaXR5MRwwGgYDVQQKDBNEZWZhdWx0IENvb XBh ............................................................. ... ............................................................. ... ............................................................. ... ............................................................. ... ............................................................. ... q7YfP2PvtiLalyYasfQoGv6h6bYYO+6uKo6OjRuonYQPp5XOCusbdUjSFlrUD iW4 OMlWn1ow9qJnEfoWKm3Og62TII3bbw8eHQTQOSlFCq7B -----END CERTIFICATE----- → → → → → → → → 3. Example of log-transformer-syslog.key The log-transformer-syslog.key alongside log-transformersyslog.crt is used to create the secret eric-log-transformerasymmetric-secret. -----BEGIN RSA PRIVATE KEY----MIIEpQIBAAKCAQEAvMVO9A9yRNVSRBZHXhij2qIk/Kqwi0h1GnuaBxuWvehQb → GgW t8szW/a0Pg4BcKEasj2TELlV5KbIRiRM6iSpe6yIp4IVOqOVxmLNNpYkrtDv+ → k4N ............................................................. → 2/1543-AOT 101 5396 Uen AL | 2022-07-25 45 EO Cloud Native Security User Guide ... ............................................................. ... ............................................................. ... ............................................................. ... ............................................................. ... N2jKxJkTdJqex7L6Ok3kO9ZJ6U6QSiGiwtEtilXtR+it1FSDWZlSLU8+D65hJ kAu nI2NU7fXdhpgerM2RLa4iLJ8j6JUhnI4atL6dIaE4NeboeF5C3+WJGg= -----END RSA PRIVATE KEY----- 6.3 → → → → → Procedure to Check for Certificate Expiry Date Use this procedure to check the validity of the certificates used in the ongoing execution of the EO Cloud Native Applications. The commands in the following examples must be run from a client where the kubectl command line is configured to access the respective cluster and namespace. It is assumed that you have the openssl command available. Steps 1. List all the secrets in the namespace: $ kubectl get secrets -n <namespace> For more details on the specific certificates and corresponding kubernetes secrets, see Certificates in EO Cloud Native Applications on page 34. 2. View the validity of a certificate: $ kubectl get secret <secret-name> -n <namespace> -o jsonpath="{.data['tls\. → crt']}" | base64 -d | openssl x509 -noout <-dates or -enddate> (-dates to show both dates) (-enddate to show the date when the certificate expires) Example $ kubectl get secrets -n test NAME DATA AGE iam-cacert-secret 1 1d iam-tls-secret s 2 1d 46 TYPE → Opaque → kubernetes.io/tl → 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Certificate Management $ kubectl get secret iam-tls-secret -n test -o jsonpath="{.data['tls\.crt']} → " | base64 -d | openssl x509 -noout -dates notBefore=Jul 20 00:00:00 2020 GMT notAfter=Jul 21 12:00:00 2021 GMT 3. If the Kubernetes secret contains more than one certificate in tls.crt attribute, for example, client type certificate secrets with multiple CA certificates, then the command in step 2 shows the expiry date of the first certificate only. In such cases, split each certificate into separate files before checking their expiry dates. a. Check if the Kubernetes secret has multiple certificates in the tls.crt attribute. The following command writes the list of certificates in the Kubernetes secret to a file called cert_bundle . $ kubectl get secret <secret-name> -n <namespace> -o jsonpath="{.da ta['tls\.crt']}" | base64 -d > cert_bundle #Example: $ kubectl get secret example-secret -n test -o jsonpath="{.data['tl s\.crt']}" | base64 -d -----BEGIN CERTIFICATE----MIIGaTCCBVGgAwIBAgIQAbmOw1rx0YblDQEOuoKlsDANBgkqhkiG9w0BAQsFADBN MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMScwJQYDVQQDEx5E ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ 4f4ippojsekQFsndbSfaQ+XieANVUEmJ8H4qBG7hatfIPi6/EGURIAa7ic3qXwec 9sjlS54Sj+9xrjS4BA== -----END CERTIFICATE---------BEGIN CERTIFICATE----MIIElDCCA3ygAwIBAgIQAf2j627KdciIQ4tyS8+8kTANBgkqhkiG9w0BAQsFADBh MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3 ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ 2/1543-AOT 101 5396 Uen AL | 2022-07-25 47 → → EO Cloud Native Security User Guide ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ ................................................................ c+LJMto4JQtV05od8GiG7S5BNO98pVAdvzr508EIDObtHopYJeS4d60tbvVS3bR0 j6tJLp07kzQoH3jOlOrHvdPJbRzeXDLz -----END CERTIFICATE----$ kubectl get secret example-secret -n test -o jsonpath="{.data['tl s\.crt']}" | base64 -d > cert_bundle $ ls . .. cert_bundle → b. If the cert_bundle file has more than one certificate, split it into individual files using the following command. Each file name begins with the prefix individual-. $ csplit -f individual- cert_bundle '/-----BEGIN CERTIFICATE-----/' '{*}' --elide-empty-files #Example: $ csplit -f individual- cert_bundle '/-----BEGIN CERTIFICATE-----/' '{*}' --elide-empty-files 2240 2301 $ ls -al total 24K drwxrwxrwt 14 root -rw-r--r-- 1 nj -rw-r--r-- 1 nj -rw-r--r-- 1 nj drwxr-xr-x 2 nj root nj nj nj nj 4.0K 4.5K 2.3K 2.2K 4.0K Aug Aug Aug Aug Aug 26 26 26 26 26 17:03 17:03 17:04 17:04 17:04 .. cert_bundle individual-01 individual-00 . c. Run the following openssl command on each individual file to check the expiry date. $ openssl x509 -in <file name> -noout -dates #Example: $ openssl x509 -in individual-01 -noout -dates notBefore=Aug 22 19:32:16 2020 GMT notAfter=Aug 20 19:32:16 2030 GMT 4. If the registry CA certificate is not present in the worker node and the complete certificate chain is added in all the worker nodes manually, execute the following steps: a. Navigate to a location in which you can access your kubernetes cluster. b. Navigate onto a worker node. c. CD into the directory with the certificate: $ cd /etc/docker/certs.d/<ingress host of container registry> d. Check the certificate expiry date: $ openssl x509 -in ca.crt -noout <-dates or -enddate> 48 2/1543-AOT 101 5396 Uen AL | 2022-07-25 → → Certificate Management (-dates to show both dates) (-enddate to show the date when the certificate expires) 6.3.1 Procedure to Check Certificate Validity in Google Chrome Use this procedure to check the certificate expiry date in Google Chrome. 1. Click the padlock icon in the address bar. 2. Click Certificate (Valid). 3. Check the validity of the certificate in the General tab of the Certificate window. 6.3.2 Procedure to Check Certificate Validity in Mozilla Firefox Use this procedure to check the certificate expiry date in Mozilla Firefox. 1. Click the padlock icon in the address bar. 2. Click the right arrow in the Site Information panel. 3. Click More Information in the Connection Security panel. 4. Click View Certificate from the Security tab of the Page Info window. 5. Check the validity of the certificate in the Certificate tab. 6.4 Procedure to Update Certificate in EO 6.4.1 Renewing a Certificate To replace a certificate in an existing installation, first delete the existing Kubernetes secret in the installation and recreate it. The commands in the following examples must be run from a client where the kubectl command line is configured to access the respective cluster and namespace. Access to an openssl command is required to run these commands. 1. Obtain a new valid SSL certificate from a trusted CA. 2. Download and extract all certificate files and create a certificate bundle according to the recommended format. You must have a certificate bundle 2/1543-AOT 101 5396 Uen AL | 2022-07-25 49 EO Cloud Native Security User Guide and a key file to renew a server type certificate. In this section, we refer to the certificate bundle as <FQDN>.crt and key file as <FQDN>.key. FQDN can be any name, but for clarity it is preferred to have the common name used in the Certificate Signing Request. To renew a client type certificate, there is no <FQDN>.key file, but only a certificate file with a .crt extension is used. For more information on the format of the certificates, see Format of the Certificates Required for EO Deployment on page 39. #Example: # For iam, if the common name (CN) is iam.ericsson.com then, certificate bun → dle is called iam.ericsson.com.crt and key file is called iam.ericsson.com. → key 3. Query and identify the secret that corresponds to the respective certificate that you are trying to renew. For more details on the specific certificates and corresponding kubernetes secrets, see Certificates in EO Cloud Native Applications on page 34. $ kubectl get secrets -n <namespace> #Example: #1. For iam.ericsson.com server certificate the corresponding kubernetes sec → ret will be iam-tls-secret #2. For iam.ericsson.com client certificate the corresponding kubernetes sec → ret will be iam-cacert-secret [Optional] Before deleting any existing Kubernetes secrets with certificates, you can take a backup of the existing certificate or key using following command. This is an optional step. The certificate content is written to a file called, certificate_file_backup.crt, and key content is written to a file called key_file_backup.key. # Taking a backup of the certificate $ kubectl get secret <secret name> -n <namespace> -o jsonpath="{.data['tls\. → crt']}" | base64 -d > certificate_file_backup.crt # Taking a backup of the key. This is only applicable for server type certif → icate secrets. $ kubectl get secret <secret name> -n <namespace> -o jsonpath="{.data['tls\. → key']}" | base64 -d > key_file_backup.key 4. Delete the respective secret from kubernetes namespace. <secret name> is identified in the previous step. $ kubectl delete secret <secret name> -n <namespace> #Example: # Deleting iam-tls-secret from eo namespace $ kubectl delete secret iam-tls-secret -n eo 5. Create the secret with the new certificate and key files obtained in step 2. For a client type certificate, the command is different as the <FQDN>.key is not available. <secret name> is the same name of the kubernetes secret deleted in previous step. A client type certificate secret can contain one or more CA certificates. 50 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Certificate Management #Creating kubernetes secret with server type certificate, including a bundl → e and a key. $ kubectl create secret tls <secret name> -n <namespace> --key <FQDN>.key -- → cert <FQDN>.crt #Example: #Creating iam-tls-secret in eo namespace. $ kubectl create secret tls iam-tls-secret -n eo --key iam.ericsson.com.key --cert iam.ericsson.com.crt → If you are renewing a client type certificate, ensure that the CA certificate you are trying to renew and any other CA certificates that exist in the client type certificate secret before deletion, are present within the .crt file used with the next commands. If you have taken a backup of the Kubernetes secret in step 3, use the certificate_file_backup.crt file to retrieve any other CA certificates which were present within the Kubernetes secret, other than the one that you are going to modify. In the following example, the file name is assumed to be intermediateca.crt and it is assumed to be located within the same directory where the command is run. The intermediate-ca.crt file must be renamed to tls.crt before it is used with the kubectl command to create the secret. #Renaming the intermediate-ca.crt file to tls.crt $ mv intermediate-ca.crt tls.crt #Creating kubernetes secret with client type certificate, without key file. $ kubectl create secret generic <secret name> -n <namespace> --from-file=tls.crt=./tls.crt --from -file=cacertbundle.pem=./tls.crt #Example: #Creating iam-cacert-secret in eo namespace. $ mv intermediate-ca.crt tls.crt $ kubectl create secret generic iam-cacert-secret -n eo --from-file=tls.crt=./tls.crt --from-file =cacertbundle.pem=./tls.crt → → 6. For a server type certificate secret, after the new secret is created in step 5, the ingress controller loads its configurations with the new certificate details. Run the following command to verify the new certificate details. It must display the "Issuer", "Subject" and "Issue Date" and "Expiry Date" for the certificate. This information in the console output must be valid and match the information of the new certificate you obtained from the trusted CA. $ openssl s_client -showcerts -servername <server endpoint> -connect <server → endpoint>:443 < /dev/null | openssl x509 -noout -subject -issuer -dates #Example: #Verifying certificate details for iam.ericsson.com endpoint openssl s_client -showcerts -servername iam.ericsson.com -connect iam.ericss → on.com:443 < /dev/null | openssl x509 -noout -subject -issuer -dates subject=C = IE, ST = Westmeath, L = Athlone, O = IAM, OU = Security, CN = ia → m.ericsson.com, emailAddress = admin@iam.ericsson.com issuer=C = IE, ST = Westmeath, L = Athlone, O = Internal CA Ericsson, OU = S → ecurity, CN = internalca.com, emailAddress = admin@internalca.com notBefore=Jul 27 08:25:49 2020 GMT notAfter=Jul 22 08:25:49 2021 GMT 7. Some Server certificates and all Client certificates can be mounted to different pods in EO, as volume mounts. When in such situations, based on the certificate that is renewed, restart these pods to ensure that correct certificate is mounted again to the respective pods from new kubernetes 2/1543-AOT 101 5396 Uen AL | 2022-07-25 51 EO Cloud Native Security User Guide secrets that are created in step 5. The running pods can be deleted to restart the pods. $ kubectl delete pod <pod name> -n <namespace> The following table provides the details about the respective kubernetes secret with certificate and the pods that require a restart: Table 22 Kubernetes Secret Name iam-cacert-secret Associated Kubernetes Pods — eric-eo-api-gateway — eric-eo-usermgmt — eric-eo-tenantmgmt — eric-eo-enm-adapter — eric-eo-so-l3nm-netfusionadapter — eric-am-onboarding-service — eric-adp-gui-aggregator-service — eric-sec-access-mgmt registry-tls-secret eric-am-onboarding-service helm-registry-tls-secret eric-am-common-wfs eric-eo-evnfm-nfvo No pods need restart. eric-eo-nfvo-cacert-secret — eric-eo-ecm-adapter — eric-eo-ecmsol005-adapter — eric-eo-sol005-adapter 52 eric-eo-cm-nbi-tls-cert No pods need restart. eric-log-transformer-asymmetricsecret eric-log-transformer eric-log-transformer-trustedexternal-secret eric-log-transformer 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Certificate Management 6.4.1.1 Renewing Certificates Used for Exporting Logs to Remote Syslog Server This section describes how to update the contents of the secrets used for exporting logs to remote Syslog server. The following are the two secrets needed to export logs to a remote syslog server: — The secret eric-log-transformer-asymmetric-secret. — The secret eric-log-transformer-trusted-external-secret. To renew the contents of the secreteric-log-transformer-asymmetricsecret, follow the instructions from section Renewing a Certificate on page 49, using the files log-transformer-syslog.key and log-transformersyslog.crt before restarting the eric-log-transformer pod. To renew the certificate used in the secret eric-log-transformer-trustedexternal-secret, do the following steps: Steps 1. Create a file named "trustedcert" with the contents of the new certificate. 2. Delete the secret as shown in Step 4 in Section 6.4.1 section. 3. Create an opaque secret from the "trustedcert" file created in Step 1 using the following command. kubectl create secret generic eric-log-transformer-trusted-external-secret - → n <namespace> --from-file=trustedcert 4. Restart the eric-log-transformer pods as shown in Step 7 in Section 6.4.1 section. 6.5 Certificate Information for PF There are three certificates that are used within Policy Framework: — Policy Framework TLS Certificate — DMaaP TLS Certificate — Identity and Access Management TLS Certificate All the mentioned certificates must be created before initiating the Policy Framework install process. 2/1543-AOT 101 5396 Uen AL | 2022-07-25 53 EO Cloud Native Security User Guide Policy Framework TLS Certificate The Policy Framework TLS certificate is used to enable secure HTTPS communication between a browser and the Policy Framework application. DMaaP TLS Certificate The DMaaP TLS certificate enables secure HTTPS communication between a browser and the DMaaP application. Identity and Access Management TLS Certificate The Identity and Access Management certificate enables secure HTTPS communication between a browser and the Identity and Access Management service with Policy Framework. The user must create and maintain the certificates. See EO Cloud Native Installation Instructions to learn how to create the certificates. 6.6 Certificate Information for SO There are three certificates that are used within Service Orchestration: — Service Orchestration TLS Certificate — Universal Design Studio (UDS) TLS Certificate — Identity and Access Management TLS Certificate — Intermediate CA certificate All the mentioned certificates must be created before initiating the Service Orchestration install process. Service Orchestration TLS Certificate The Service Orchestration TLS certificate is used to enable secure HTTPS communication between a browser and the Service Orchestration application. Universal Design Studio TLS Certificate The Universal Design Studio TLS certificate enables secure HTTPS communication between a browser and the Universal Design Studio application. Identity and Access Management TLS Certificate The Identity and Access Management certificate enables secure HTTPS communication between a browser and the Identity and Access Management service with Service Orchestration. Intermediate CA certificate 54 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Certificate Management Service Orchestration needs to communicate with different subsystems (NFVO, Domain Manager, Network Controller, PNF). Ensure that the CA certificate of each subsystem must be present within the iam-cacert-secret certificate content so that the HTTPS protocol can be used when Service Orchestration is communicating with these onboarded subsystems. To do this, append the CA certificates of each subsystem to the intermediate-ca.crt file before or after the installation or upgrade. 6.6.1 Add SO Subsystem Certificate To add SO subsystem certificate after SO installation or upgrade: Steps 1. Obtain the subsystem's CA certificate. 2. Export the existing certificates in the secret to a file: kubectl get secret iam-cacert-secret -n <namespace> -o jsonpath="{.data['tls\.crt']}" | base64 -d > tls.crt 3. Edit the resulting tls.crt file and append the subsystem's CA certificate, obtained in step 1. 4. Save and close the editor. 5. Delete the old secret: kubectl delete secret iam-cacert-secret -n <namespace> 6. Recreate the secret using the updated tls.crt file: kubectl create secret generic iam-cacert-secret -n <namespace> --from-file=tls.crt=./tls.crt --from-file=cacertbundle.pem=./ tls.crt 7. Restart the subsystem's adapter pods by deleting them – Kubernetes will recreate them: kubectl delete pods $(kubectl get pods -n <namespace> | grep "adapter" | egrep "ecm|sol005" | awk '{print $1}') -n <namespace> --force --grace-period=0 8. Wait until all pods are back to Ready state. 6.7 Certificate Information for UDS There are two certificates used within Universal Design Studio — Universal Design Studio (UDS) TLS Certificate 2/1543-AOT 101 5396 Uen AL | 2022-07-25 55 EO Cloud Native Security User Guide — Identity and Access Management TLS Certificate All the mentioned certificates must be created before initiating the Universal Design Studio install process Universal Design Studio TLS Certificate The Universal Design Studio TLS certificate enables secure HTTPS communication between a browser and the Universal Design Studio application. Identity and Access Management TLS Certificate The Identity and Access Management certificate enables secure HTTPS communication between a browser and the Identity and Access Management service with Service Orchestration. The user must create and maintain the certificates. 6.8 Certificate Information for EO CM Note: 56 EO CM GUI/NBI will be unavailable when the secret eric-eo-cm-nbi-tlscert is being re-created with a new certificate. 2/1543-AOT 101 5396 Uen AL | 2022-07-25 Reference List Reference List [1] [2] [3] [4] EO Glossary of Terms and Acronyms, 0033-AOT 101 5396 Typographic Conventions, 3/1551-FCK 101 05 EO Cloud Native Installation Instructions, 1531-AOT 101 5396 Available in EO Cloud Native Support Product Information Library, EN/LZN 703 0279 Identity and Access Management, EN/LZN7020496 Available from local Ericsson support. 2/1543-AOT 101 5396 Uen AL | 2022-07-25 57