Just 'cos I said it, doesn't mean it's right
Creating your first Zenpack? Me too.
I am finding these resources very helpful:
They are pretty dry, so have coffee on hand. You can always pause if you need a break.
If you are trying to collect performance data from IBM’s Tivoli Directory Server, and you do not have Tivoli Directory Integrator installed, then you can still monitor some performance metrics with Zenoss.
The big Aha! moment for me was when I read that you can query some useful metrics with an LDAP client.
ldapsearch -h $LDAP_HOST -x -D "$LDAP_ADMIN_DN" -w $LDAP_ADMIN_PASSWORD -s base -b cn=monitor objectclass=* # extended LDIF # # LDAPv3 # base <cn=monitor> with scope baseObject # filter: objectclass=* # requesting: ALL # # MONITOR dn: CN=MONITOR version: IBM Tivoli Directory (SSL), 6.1 totalconnections: 50912 total_ssl_connections: 0 total_tls_connections: 0 currentconnections: 67 maxconnections: 1024 writewaiters: 0 readwaiters: 0 opsinitiated: 712582 livethreads: 1 opscompleted: 712581 entriessent: 620628 searchesrequested: 585265 searchescompleted: 585264 bindsrequested: 50917 bindscompleted: 50917 unbindsrequested: 50791 .......
So we no longer need to use IBM’s SNMP listener therefore saving some time and maybe even some money. Zenoss allows you to run scripts and so long as the script returns stuff in the right format Zenoss can graph them. Here’s how I did it.
First we need a script to go get the data from TDS. As shown above, it is really only a simple LDAP search, but the output need to conform to the Nagios plugin standard. So here is my script. Feel free to use and improve upon it.
#!/bin/bash
# Robert Hart https://robertjhart.wordpress.com/ July 2010
# Script to collect performance metrics from IBM Tivoli Directory Server 6.1
# For reference:
# http://community.zenoss.org/docs/DOC-4770
# http://nagiosplug.sourceforge.net/developer-guidelines.html#PLUGOUTPUT
# List of attributes to get.
# format: space seperated - attribute,UOM
ATTRIBUTES="bindsrequested,c currentconnections,"
# Stuff to tweak
LDAP_HOST=$1
LDAP_ADMIN_DN=<administrator distinguished name>
LDAP_ADMIN_PASSWORD=<password>
CMD="ldapsearch -h $LDAP_HOST -x -D "$LDAP_ADMIN_DN" -w $LDAP_ADMIN_PASSWORD -s base -b cn=monitor objectclass=*"
TMP_FILE="/tmp/$$"
# if needed, set to a real file
DEBUG="/dev/null"
# this parses the attribute value out of the ldpsearch output.
# usage: parse attribute,UOM
function parse
{
echo "parsing $1" > $DEBUG
ATT=`echo $1 | awk -F "," '{print$1}'`
UOM=`echo $1 | awk -F "," '{print$2}'`
KEY=`grep "^$ATT" $TMP_FILE | awk '{print $1}' | sed -e 's/://'`
VALUE=`grep "^$ATT" $TMP_FILE | awk '{print $2}'`
if [ -z $UOM ]
then
OUTPUT="$KEY=$VALUE;;;; $OUTPUT"
else
OUTPUT="$KEY=$VALUE[$UOM];;;; $OUTPUT"
fi
}
# Lets do some work
if $CMD > $TMP_FILE
then
echo "command ran" > $DEBUG
else
exit 2
fi
for ATTRIBUTE in $ATTRIBUTES
do
if parse $ATTRIBUTE
then
echo "parsed" > $DEBUG
else
exit 1
fi
done
echo "tds |$OUTPUT"
rm $TMP_FILE
exit 0
Save this script as a file on the Zenoss server, make the zenoss user the owner and give it execute permissions.
You should be able to test the script and get a result:
$ ./zenoss-tds.sh ldap.example.org tds |currentconnections=69;;;; bindsrequested=50880[c];;;;
First I created an device class because we have a couple of LDAP servers, and so devices inheriting from the device class is the more efficient way to do this.
I also set the LDAP monitor zProperties and bound the LDAP monitor template so we could graph LDAP response times too.
In the templates tab for the device class, I pulled down the menu in the Available Performance Templates section and selected “Add Template…”. Once you have given it a name, then you end up in a page where you can add a data source. In the Data Sources section pull down the menu and select “Add Datasource…”. Give it a name, and set the source to COMMAND. Make sure you set the parser to Nagios, and make sure you pass the device name to your command, eg. /opt/zenoss/scripts/zenoss-tds.sh ${devname}. Click Save, and then add DataPoints at the bottom.
When you create the DataPoints, remember to set the correct Type. Since most of the metrics in TDS zero themselves when you restart the server, then COUNTER is probably the most appropriate.
Once you have done that then you can go back to the template and add graph definitions. Then you can attach the appropriate data point to the graph.
Lastly you need to bind your new template to the device class. You do that in the templates tab again. Remember to hold down the control key or you will deselect all the other templates in use here.
If you have not already done so, you can put a device in to that class and odel. You should start to see data for the metrics you are collecting.
And thanks to Dan for his help too.
I notice that my notes from a couple of days ago gets a lot of hits, so I feel inclined to be a bit more detailed about how to integrate Ubuntu Lucid Lynx in to a windows domain.
This is worth doing regardless of whether or not you set up PAM. You can see why later.
sudo apt-get install krb5-config krb5-user
During the installation process, it will ask you for your realm. Enter the Realm for your Windows Domain (talk to your Active Directory administrator if you don’t know)
You will need to add a stanza for your realm in /etc/krb5.conf. something like this:
<REALM> = {
kdc = <active directory server>
admin_server = <active directory server>
default_domain = <domain.name>
}
kinit <user>
The user should be a windows domain user. When challenged input the windows password for that account. Look for the ticket with the klist command.
If you want to sign in to your desktop / server using your Windows network credentials, then do follow these steps.
Set up the root account. If you mess this up and you need to fix it, then you need to be able sign in as root. I strongly recommend you do this so that you can.
sudo passwd root
and then test:
su -
Glad we got that done – lets move on.
sudo apt-get install libpam-krb5
Ubuntu sets up PAM for you, so that should be it.
Try to log in to the computer with your windows credentials. There must be a local account already created and the user name and must be the same as the domain user name. It should just work. Run klist and you should also see a ticket. Nice!
If you have kerborized web applications, or spnego enabled sites, then you can configure Firefox to use your kerberos ticket to negotiate for you and logging you in unchallenged. You need to type the following in to the location bar (preferably in another tab – you don;t want to lose this page just yet)
about:config
Say that you will be careful and make sure the following attributes are set to true:
Set the following to your local DNS sub-domain. This defines the scope of the trust for sites it will try to negotiate with:
Next time you attach to such a web site, then you will get logged in.
If your company uses the Microsoft Office Communicator Suite for IM, then you too can join the conversaton with the Sipe plugin. Install it with this commend:
sudo apt-get install pidgin-sipe
Once you have restarted Pidgin, then you can add an account. Here are some guidelines:
Kerberos seemingly works but the version that is here has not really worked for me. Give it a try by blanking out the password and checking the kerberos box and see how you do.
Once you have a kerberos ticket, then you can mount file shares without providing a user name or password. You can use the connect to server form in the Places menu. You set the service type to be windows server, set the server to the fully qualified domain name of the windows file server and enter the share name. You do not need to put a user name of domain in. Bookmark if you like. The share should open up in nautilus with no further prompting.
I tried the evolution-mapi plugin which implements the Exchange MAPI protocol. It works but I found it sluggish and still very buggy. I would wait a little longer for anything more serious than testing.
That’s all folks!
If you have any other tricks that I have not mentioned then let us know.
Ubuntu 10.04, the Lucid Lynx is now released to the wild. There is lots of stuff out there talking about all the cool stuff that is included. I am going to talk about some of the stuff that you probably won’t see in the reviews.
A fairly niche subject important to those in a Kerberos environment, or who want better integration in a Windows Domain. In Ubuntu 10.04, setting up a kerberos client just got a lot easier.
When you install the krb5-config package, it will ask you some questions abot the realm you are in, etc. It does not do everything, but it does most of the work. I still had to edit /etc/krb5.conf to add in the hostname of the KDC, etc.
Installing libpam-krb5 does the right things to configure PAM. You can start to use it straight away, and it just works. For me, it also creates a kerberos ticket for you, which I don’t think it did before. This, to me, is a big deal. It means that I can use firefox to go into kerborized, and spnego enabled web sites without having to manually create a ticket before-hand. Same with kerborized ssh servers, and pidgin-sipe.
I tried libpam-ccreds too and it also just worked. Again, no messing with PAM configurations.
If you are in a Windows Domain and need to IM with your colleagues who are hanging out on Office Communicator, then pidgin-sipe does the job very well. If you hover over a buddy icon, then you can see what is on their calendar now. Pidgin also then sets your status according to what is on your calendar, so if you are scheduled for a meeting then it will set your status to busy at that time.
Once you have your kerberos ticket, then you can mount cifs file shares in the domain by running:
gvfs-mount smb://server-fqnd/share
The share is then set up in GNOME and you are not challenged for credentials. That means that with libpam-krb5 and a login script, you can have all your Windows shares auto-magically mounted when you log in. Nice!
Very often the discipline of documentation does not come naturally to systems administrators. When it comes to aspects like relationship data, this is not immediately obvious to them, and the benefits of capturing and sharing that data is probably a little esoteric. This means that without a direct and immediate benefit to them, getting SA’s to input this data is going to be a chore.
The social networking paradigm would mean that entering data is a little more fun and interesting than filling the forms you see in most configuration management databases. Hopefully this would be an added motivator to keep data up to date.
In my last post I extolled the virtues of mapping relationships between configuration items. As I was thinking about this more, it came to me that there are many similarities between this and social networking. In both we map and track relationships and in both we use these relationships to derive value and aggregate information.
In my mind the social networking product that best fits the analog is Facebook. You create a profile for yourself, you build up a network of relationships, and you have a stream of status information. Your friends aggregate this status information to form a single news feed about all the people they care about.
So in terms of a configuration management equivalent, we have a profile for each server, application and service in our data center. We then link them together with relationships. I would add more information with these links like what type of link it is. Because servers don’t have the same privacy concerns we do, then I would have the page in the profile that shows the relationships map recursively as far as they can up and down the pyramid stating who is related to what and how. Each item that is mentioned should be hyper-linked to their own profile page for easy navigation.
The “Wall” as it were can be an aggregation of status messages from the incident/problem/change ticketing system, and maybe other areas like the network monitoring and performance monitoring tools that may be deployed. I think it would be really cool for system administrators to be able to reference items in a micro-blogging environment which then magically appear on the wall.
No I am not talking about human relationships – although they matter too. I am talking about configuration management. Let me explain.
First of let me clarify configuration management. I am not talking about enforcing standard server builds and standard software configurations. I am talking about getting all the stuff in your data center in a database and being able to use that information. This is more like the ITIL view of configuration management.
The ITIL standard states that you should keep records of all “configuration items”, accurate details about them and also data like relationships between CIs and change history.
The first part just sounds like simple asset management to me and once your data center has grown to a certain size, then you are probably pretty good at that. It seems to me that the next level is relationship mapping and I am pretty sure that is where you get the most bang for your buck when it comes to ITIL CMDBs.
To illustrate my point, lets build a word picture of how a set of relationships might look:
Now this is a very simple set of relationships and already we can draw some very useful insights:
Immediately, we can see that there is huge value to relationship data. This is why it matters and why it is worth maintaining this data. Once this data is being collected, maintained and used, then in my mind that is a big milestone towards transitioning your systems team from an asset centric operations oriented shop to a more service oriented, and hence customer focused endeavor.
So, what has caught my attention recently?
The fruits of another evenings youtube watching.
I found myself pulling this all together from various places. I actually did it a couple years back so somebody else may have put up a better howto by now.
The end result here will be a samba server on a Windows domain acting as a member server with all the ACL stuff, etc working.
This was developed on RHEL. I am sure it will work on CentOS with little to no modifications and Debian based distros will need to make some changes.
There are bound to be mistakes and omissions, If you see something wrong then let me know.
Setting this up enables a few things to be possible. In this instance it enables you to generate a keytab file. Then you can use SASL as the lauthentication method when doing LDAP queries against Active Directory. LAter we may talk about setting up PAM for loggin on to the server with your Windows username/password, or creating UPN’s for things like webservers.
/etc/krb5.conf
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log
[libdefaults]
default_realm = <YOUR.REALM>
dns_lookup_realm = true
dns_lookup_kdc = true
forwardable = true
proxiable = true
[realms]
<YOUR.REALM> = {
kdc = <active.directory.server>
admin_server = <active.directory.server>
}
This is so that you can get user and group information from active directory. I should point out that the users and groups that will be used for this must have POSIX attributes set up in the objects on the directory.
/etc/ldap.conf
base <search base of active directory? uri ldap://<active.directory.server> referrals no ldap_version 3 bind_policy soft # "getent passwd" will hang if this isn't set for AD sasl_secprops maxssf=0 use_sasl on rootuse_sasl on pam_filter objectclass=user pam_password ad
If you want to restrict the search bases for various searches, add lines similar to these in ldap.con.
nss_base_passwd <search base dn> nss_base_shadow <search base dn> nss_map_objectclass posixAccount user nss_map_objectclass shadowAccount user nss_map_objectclass posixGroup group nss_map_attribute username uid nss_map_attribute homeDirectory unixHomeDirectory
This file needs to be set up to get you on to the Domain and for keytab file creation. LAter, your share definition will go in here too.
/etc/samba/smb.conf
security = ads
realm = <YOUR.REALM>
password server = <active.directory.server>
username map = /etc/samba/smbusers
use kerberos keytab = yes
ldap ssl = yes
encrypt passwords = true
use spnego = yes
obey pam restrictions = no
invalid users = root
This tells the server which directories to search and in which order when looking for stuff like user or group information. In this situation, we are using LDAP to query AD.
/etc/nsswitch.conf
passwd: files ldap shadow: files ldap group: files ldap .... hosts: files dns wins
As root:
net -U <user> ads join createupn="HOST/<host>@<YOR.REALM>" createcomputer="<Active/Directory/server/account/container>"
For an explanation of the above commend (because it looks confusing). See this quote from the net man page:
[RPC|ADS] JOIN [TYPE] [-U username[%password]] [createupn=UPN] [createcomputer=OU] [options] Join a domain. If the account already exists on the server, and [TYPE] is MEMBER, the machine will attempt to join automatically. (Assuming that the machine has been created in server manager) Otherwise, a password will be prompted for, and a new account may be created.[TYPE] may be PDC, BDC or MEMBER to specify the type of server joining the domain.
[UPN] (ADS only) set the principalname attribute during the join. The default format is host/netbiosname@REALM.
[OU] (ADS only) Precreate the computer account in a specific OU. The OU string reads from top to bottom without RDNs, and is delimited by a ‘/’. Please note that ‘´ is used for escape by both the shell and ldap, so it may need to be doubled or quadrupled to pass through, and it is not used as a delimiter.
Next create the host keytab file:
net ads kytab create -U <username>
chkconfig smb on
servic smb start
Winbind is optional in this configuration. If you do not use it make sure you do not include it for host name resolution in /etc/nsswitch,conf.
chkconfig winbind on
service winbind start
To test user and group resolution over ldap:
getent passwd <user>
getent group <group>
If things are working then you should see AD users and groups as will as local.
If you are using winbind:
wbinfo -u
wbinfo -g
These commands may be unreliable in large domains. Try a couple of times since it may take some time to download the databases.
If all this works, then you can use AD users and groups in file AC:’s
/etc/samba/smb.conf
[share]
comment = some share
path = /path/to/share
public=no
browseable = yes
writable = yes
We can use AD groups in posix ACL’s to control file and directory access. This allows us to do stuff like set up “default” acl’s that are inherited when a child object is created. Remember that in samba (unless told otherwise) rwx in linux = Full control on windows. That probably is not what a Windows power user expects. AFAIK you cannot yet implement “TAke Ownership” and the other Windows specific File permissions because they do not exiasst in Linux.
in a nutshell:
View ACL by: getfacl file Set using setfacl. One option is to pipe the output of getfacl to a file, edit the file and
setfacl -M < file Or you can use -m to modify the acl inline. Use -R to make the change recursively.
Examples:
setfacl -R -m default:group::rx /opt/directory #generic group permission that is inherited by child. set recursively setfacl -m user:rhart:rwx /opt/directory/file #give full control to rhart for the specified file
Robert Hart 