Sending Data to Oracle Management Cloud

Sending Data to Oracle Management Cloud

Lately I’ve been busy helping customers deploy the new Oracle Management Cloud (OMC) services and discover what it can do for them.   One of the most frequent questions I get when talking with new customers about OMC is how do we collect the data and send it to Oracle’s Cloud?  Using Agents installed on-premises or in the Cloud, we can collect various points of data to be used in the many services that make up Oracle Management Cloud.  The goal of this blog is to introduce you to our points of communication and explain where we get data and how it’s used.

It’s important to note, all communication is outbound from Agents to Oracle Cloud.   There is no inbound communication.  All agent communication is over HTTPS.  If your systems have direct access to the internet, they can communicate directly.  Most enterprise customers will likely configure a proxy or firewall/ACL changes to allow communication over certain servers and ports.

Each tenant will have a unique URL that the agents would access, typically it would look like:

https://<tenant>.itom.management.us2.oraclecloud.com/*

Example:
https://tenant1.itom.management.us2.oraclecloud.com/registry

OMC Architecture

Agents

Cloud Agent

The Cloud Agent is responsible for collecting log, monitoring and performance data to be used in the various services (Log Analytics, IT Analytics and Infrastructure Monitoring).  This agent will sit on the target host, whether on-premises or in a Cloud.  The agent can communicate to OMC directly, via a proxy, or the Gateway (to be discussed below).

APM Agent

When using Application Performance Monitoring (APM), you’ll be using the APM agent which is installed on the server where the application runs. Whether the application is Java, .Net or Node.js, the agent collects and sends performance data about the application to OMC.  This agent can communicate directly, via a proxy, or through the Gateway.  As APM also collects End User Metric data from the application users browser, that data is sent directly to OMC.

Gateway

The Gateway is an optional component and is used to buffer and send all data from Cloud Agents, APM Agents and Data Collectors to the Oracle Cloud.   This way, only one server has to have internet access.  Due to the amount of activity for buffering data, this may require a standalone machine as the CPU utilization can be up to 20% for large environments depending on the number of agents and volume of data being sent.

Data Collector

There is also an optional Data Collector component for users who already have Oracle Enterprise Manager (OEM) configured.  The Data Collector extracts target properties, associations, metrics, incidents, version and additional configuration data about the targets in OEM and shares this with OMC.   The Data Collector talks to OMC through the Gateway.  You can have multiple Data Collectors if you have more than one OEM configured.  This is a great opportunity to consolidate data into one place if your OEM is separated by organization, geography or lifecycle status.

Deployment

Installing these agents is easy.  Customers download a master install kit from  OMC which is used for all installs.  The install kit contains tenant information and a registration key is provided during install to authenticate your agents.

A sample installation command for installing the Cloud Agent communicating through a Gateway:

$ ./AgentInstall.sh AGENT_TYPE=cloud_agent AGENT_BASE_DIR=/u01/app/oracle/omc/cloud_agent GATEWAY_HOST=hostname.server.com GATEWAY_PORT=1840 AGENT_REGISTRATION_KEY=<tenant specific> 

Downloading lama agent software ...
Generating emaas.properties ...
Extracting Agent Software ...
Installing the Agent ...
Registering the Agent ...
Downloading Certificates ...
Configuring the Agent ...
Cleanup temporary files ...

The following configuration scripts need to be executed as the root user
#!/bin/sh
#Root script to run
/u01/oracle/omc/cloud_agent/core/1.12.0/root.sh

Maintaining Agents

Since Oracle Management Cloud is an agile product with monthly releases, the agents may be updated monthly as well.  This does not mean you have to upgrade your agents monthly, but upgrading quarterly will keep your agents in top shape.   When you’re ready to upgrade your agent, you simply go to the Agents administration page and select Upgrade.  This will instruct your agent to download the latest agent and perform an out-of-place upgrade seamlessly.  If the upgrade fails for some reason, your original agent will remain active.   There is also an option to batch upgrade your Agents.

Summary

As you can see, communication to Oracle Management Cloud has a couple of options to meet different needs.  All communication will go over https and can communicate directly to the cloud or through a centralized gateway.  If you’re looking for additional information, you may find these sources helpful:

Changing OMS or Agent Properties in OEM Console

There was a thread going on earlier this week about how to change a value for an OMS property setting.   This is typically done when working with support to adjust a timing or enable debug or tracing.

The most common way is using emctl set property at command level.

$ emctl set property -name oracle.sysman.eml.maxInactiveTime -value 60 

To get the current setting there’s a emctl get property command:

$ emctl get property -name oracle.sysman.eml.maxInactiveTime

That usually works great if support has just given you the exact command to run, or you’re using a MOS note to reference the exact syntax.   However, if you’re getting older like me, and syntax is just one of those things that you tend to put in the way back corners of your brain… you tend to forget was it oracle.sysman.eml.maxInactiveTime or was it oracle.em.sysman.maxTimeout or…   There’s just too many to remember.   Lucky for us, there is now a place to view and set these properties in the EM  console.   You can find it under Setup / Manage Cloud Control / Management Services.

properties_1

Then under the Management Servers menu select Configuration properties.

properties_2

From here you’ll get a window that lists the non-default properties.  Understand, that some properties will not show up, that doesn’t mean they are not set, but that they just have the system default value.

properties_3

 

By switching the Show view to All, you’ll see a larger list of properties.  Not all of them are modifiable, as indicated by the lock icon.   If you want to view more information about a property, or modify it, click on the Name.    This will bring up a new window, with the ability to modify the value and save.  This view will also tell you whether the property is Dynamic (can be changed without OMS restart).   If you expand the Change History, you can also view the previous changes for this parameter.

properties_4

Of course, not all properties are OMS based, so there’s an equivalent option on the Agent side.  From the Agent home page, click on Agent menu then select Properties.

properties_9

You will get a list of properties, some which you can edit, some you can’t.   By default the basic properties are shown.

properties_10

Select Advanced Properties to see additional agent properties such as dynamicPropsComputeTimeout which is often adjusted on very large servers.

properties_11

This is great right?  But what if you want to change the same property on 1000 agents?   Well, that’s in here too!  Click on Setup / Manage Cloud Control / Agents.

properties_5

From there, click on the agents (or just one for now) and click Properties.properties_6

This will start a job wizard in which you can add additional agents by clicking Add in the Targets section.  Then click on the Parameters tab.

properties_7

Now you can set the parameter value that you wish to push out to all selected agents.

properties_8

The caveat — use with caution and common sense.  There’s a lot of parameters in here, and very little are documented, some should not be changed unless directed by Oracle Support.   So don’t go cowboy on us and start tweaking them all just to see what they do!

Resolving Conflicts While Patching the EM Agent

Unless you’re in a cave asleep, you’ve seen the recent Oracle PSUs and patches that were released in January.  This has many of my customers patching their agents, and a few have noticed a problem with some previously applied patches.  Thanks to Brian for pointing this out!

If you applied the recommended Java patches sometime last year (18502187, 18721761), and you go to apply the 12.1.0.4.5 agent patch 20282974 (or an earlier bundle), the analyze step will fail as the Java patches are included in the agent bundle, but for some reason it can’t “ignore” them.    Checking the detailed results of the failed Analyze job will show which step failed.   In this specific case, the step PrerequisiteCheckForApply will be marked Failed and the error will look like this:

ap1

To remedy this with EM, you can create a patch plan with the Java patches (18502187, 18721761) and select Rollback in the Deployment Options.  This will run the rollback for these patches.

ap2

Once rolled back, re-analyze your 12.1.0.4.5 patch plan and it should be successful and allow you to deploy!

When you update Agents, keep these things in mind:

  • Be sure to update any Agent clones/gold images with the newly updated patch and plug-ins if they’ve been upgraded or patched.
  • If you staged patches for auto-deployment in the OMS $ORACLE_HOME/install/oneoffs, remove the old patches and you should just need the 12.1.0.4.5 patch now, or any current Discovery patches.
  • Update your procedural documents to reflect the current patches required.

For more details on how to patch your agent using EM, see the Administrator’s guide.

 

Simplified Agent and Plug-in Deployment

On your site of hundreds or thousands of hosts have you had to patch agents immediately as they get deployed?  For this reason I’ve always been a big fan of cloning an agent that has the required plug-ins and all the recommended core agent and plug-in patches, then using that clone for all new agent deployments. With EM 12c this got even easier as you can now clone the agent using the console “Add Host” method. You still have to rely on the EM users to use the clone.The one problem I have with cloning is that you have to have a reference target for each platform that you support. If you have a consolidated environment and only have Linux x64, this may not be a problem. If you are managing a typical data center with a mixture of platforms, it can become quite the maintenance nightmare just to maintain your golden images.You must update golden image agents whenever you get a new patch (generic or platform specific) for the agent or plug-in, and recreate the clone for each platform. Typically, I find people create a clone for their most common platforms, and forget about the rest. That means, maybe 80% of their agents meet their standard patch requirements and plug-ins upon deployment, but the other 20% have to be patched post-deploy, or worse – never get patched!

While deployed agents and plug-ins can be patched easily using EM Patches & Updates, but what about the agents still getting deployed or upgraded? Wouldn’t it be nice if they got patched as part of the deployment or upgrade? This article will show you two new features in EM 12.1.0.3 (12cR3) that will help you deploy the most current agent and plug-in versions. Whether you have 100s or 1000s of agents to manage, reducing maintenance and keeping the agents up to date is an important task, and being able to deploy or upgrade to a fully patched agent will save you a lot of time and effort.

Read original post here.