6 Replies Latest reply: Apr 29, 2010 5:58 PM by 445843 RSS

    Dynamic Group Creation/Management in Grid Control

    richard evans
      Good Evening,

      I've been researching a minor hassle we're running into. We currently have 56 Oracle DBs managed by GC and almost 300 "Targets" in total. In order to manage these we break them out into groups - "All Databases", production, development, sandbox, SAP, Non-SAP, hardware-specific, OS-specific, etc. Every time a new DB comes online it's a bear to remember to put it into the correct groups and then, because we run Solaris 10 zones, Oracle VM, etc), we may not keep the DB on the same server it was originally installed on. So I need to dynamically update these Group Targets.

      I first started researching the packages and procedures in SYSMAN. I thought I could pretty easily write a KSH script and schedule it to run ~weekly in GC Jobs against the DB and get all Targets associated with the DB's Agent. I planned to use SYSMAN.MGMT_TARGET.ADD_GROUP and then MODIFY_GROUP but I've been struggling with the odd datatypes of SMP_EMD_NVPAIR_ARRAY, SMP_EMD_NVPAIR, and MGMT_TARGET_PROPERTY_LIST. The first two I understand but the third I cannot find any docs or help on at all. I've grep'd through OMS_HOME/sysman and haven't found anything that has given me that “duh” moment yet.

      I've also looked at the SYSMAN.EM_GROUP package and it's the same thing. I can't quite tweak my syntax to work. Also, it looks like there are additional packages/procedures/functions that run in order to create the charts, etc. I realize I could trace the DB and capture most of it... but this seems like too much work. I feel like Oracle must have already developed a better way to do what I want.

      Next, I've looked at emcli and emctl. Emcli looks very promising! I'm a little concerned about deploying it across the enterprise but I can get over that. I think I'll “install” it in on an NFS mount that is common to all databases and then run setup on each individual database using that copy. But then I wondered how I would get all of the targets. I could get the DB and OS and Agent pretty easily... but we're also running Apache, OHS, MySQL, Remedy, etc. And I want this to be as dynamic as possible. So I looked at AGENT_HOME/sysman/emd/targets.xml but that's kind of hard to parse...

      emctl config agent listtargets gets me VERY close. But again, not quite there. I'll have to strip off the “[“ and “]” which is easy enough... but I'll then have to put it into a file and pass it to emcli and let it spit it @ GC. This, again, makes me question whether this is really, truly, the easiest way to do what I want to do :-)

      I've been thinking about this, off and on, for the past 48 hours and now it's time to ask the community! Does anyone have any suggestions or thoughts? Am I way out in left field here?

      I can really save us a ton of time/energy if I can pull this off. We will know exactly where our Targets are and to which Group(s) they belong. I can parse it by hostname, DB name, IP address subnet, server name (odd is production, even is development), etc.

      Hopefully it's apparent by now I'm not looking for a “Use the GUI” type of answer :-)

      Thanks in advance!

        • 1. Re: Dynamic Group Creation/Management in Grid Control
          Rob Zoeteweij
          Hello Richard,
          - I could imagine that you create a User Defined Metric that alerts as soon as a database is not included in at least one Group and even check if it is included in the correct Group(s)
          - You could use the Target Properties for a database (Comment, Contact, Deployment Type, Line of Business, Location). Let's say you use the Target Propery "Deployment Type" to determine the Group it should be assigned to.
          - Another thing you might think of is using EMCLI MODIFY_GROUP to assign a database to a Group.
          To check if a Database is in a Group, you should check it exsitance in MGMT$TARGET_COMPOSITE
          - So, perhaps using this View and the contents of Database Property "Deployment Type", you can generate a shell script that will take care of assigning Databases to the correct Group, using EMCLI.


          For more infor on using OEM GC, you might want to check http://oemgc.wordpress.com

          Please reward good answers by marking them correct or helpful...
          • 2. Re: Dynamic Group Creation/Management in Grid Control
            richard evans
            Thanks Rob,

            - Another thing you might think of is using EMCLI MODIFY_GROUP to assign a database to a Group

            Yep! This is one of the things I've been considering but getting the list of targets is a bit tricky. I can get them by executing "emctl config agent listtargets" and parsing it. I'm curious if this would be the "best practice" though.

            I'll definitely look into a UDM, that's a great idea. At least then I'll have a warning reminding me to add it to a group.

            Any other helpful hints are always welcome!

            • 3. Re: Dynamic Group Creation/Management in Grid Control
              Rob Zoeteweij
              Why not using EMCLI GET_TARGETS?
              • 4. Re: Dynamic Group Creation/Management in Grid Control
                richard evans
                Hi Rob,

                Yep, exactly! That's what I'm looking to do with emcli and emctl. I can get the targets the agent knows about and use modify_group, get_group_members, etc, to build the target group. I'll have to take into account Jobs scheduled against the target.. I don't want to have 0 members of the target group for any moment because I'll probably lose the jobs scheduled against them. I'm not sure yet... still testing.. thinking.. drinking loads of coffee :-)

                I'm just curious if this is the best solution or if there is something easier/better I should be doing.

                • 5. Re: Dynamic Group Creation/Management in Grid Control
                  I use a simple emclli statement to get the proper syntax, etc from OEM:

                  ${EMCLI} login -user=sysman -pass="${CONSOLE_PWD}"
                  echo "\n\nGetting the exact target name for $thisSID from OMS ...\n"
                  if [ `${EMCLI} get_targets -targets="oracle_database" | grep -i ${thisSID} | wc -l` -gt 0 ]; then
                       thisTARGET=`${EMCLI} get_targets -targets="oracle_database" -format="name:csv" | grep -i ${thisSID} | cut -d, -f4`
                       echo "\t${thisTARGET}\n"
                       echo "\n\nSorry, ${thisSID} database is not in the Grid Control repository\n\n\n"

                  .. then I plug in $thisTARGET to build/start and end/stop blackouts. I know you can do that with emctl but only if your emagent has the same O/S account as your database binaries.

                  You probably just care about the guts:

                  ${EMCLI} get_targets -targets="oracle_database" -format="name:csv" | grep -i ${thisSID} | cut -d, -f4
                  • 6. Re: Dynamic Group Creation/Management in Grid Control
                    Thanks for the response!

                    I ended up doing something very similar. I execute a Multi-Task Job in Grid Control that runs a few shell scripts and SQL statements to upload this data then I use an APEX Interactive Report to get the "differences." The steps are something like this:

                    1. GATHER TARGETS ON HOSTS - Korn shell script that gets the targets on each host and parses them into a SQL*Loader-friendly format. Target = All Hosts.

                    for TARGET in `$EMCTL config agent listtargets | $GREP \\\[ | $SED -e 's/\[//g' -e 's/\]//g' -e 's/ //g' | $NAWK -F, '{print $1";"$2}'`
                    # get a list of current group members
                    echo "$SERVER_NAME;$TARGET;$DATE" >> ~/GC_TARGET_MODIFICATION/.gc_target_modification.out.$DATE.$HOSTNAME

                    2. BACKUP GC TARGET UPDATE TABLE - GC SQL Script that backups the existing data in case I biff something. Target = APEX DB

                    3. TRUNCATE EXISTING TARGET DATA - GC SQL Script that preps the table for the new data... Target = APEX DB

                    4. LOAD DATA INTO APEX SCHEMA - Korn shell script that builds the file for SQLLDR and executes it. Target = APEX DB

                    cat .gc_target_modification.out.$DATE.* >> gc_target_update.out
                    sqlldr userid=schema/password control=gc_target_update.ctl log=gc_target_update.log

                    5. COUNT NEW TARGETS - GC SQL Script that gives me a warm fuzzy that things went well or a warning that it went poorly. Target = APEX DB

                    6. APEX INTERACTIVE REPORT - It looks something like this... our servers are named after the Sun hardware so it's either m4000 or m5000 hardware and a few Sun Blades. So my groups are named the same.

                    select g.server_name, m.group_name, g.target_name, g.target_type
                    from (select server_name, target_name, target_type
                    from gc_target_update) g
                    , (select group_name, target_name, target_type
                         from MGMT$GROUP_MEMBERS
                              where group_name like 'm500%'
                              or group_name like 'm40%'
                              or group_name like 'sb%') m
                    where g.target_name = m.target_name (+)
                    and (g.server_name != m.group_name
                    or m.group_name is null
                    or g.server_name is null)
                    order by g.target_name

                    I'm sure there are better ways to do this but this is what I have right now! Maybe with EM GC 11gR1 they added this feature so I don't have to hack at it ;-)