Installing Open Virtual Machine Tools is a nice alternative to managing and installing the vmware tools package that comes with Vmware’s hypervisor products.  It allows you to use APT to manage all the dependencies and updates. Module assistant will recompile the kernel modules for you automatically when you update kernels too.

I have verified that this configuration works under ESXi 4.1.0, 260247 and a new Squeeze install. All of the essential functions like being able to see guest OS information on the Vsphere summary page and performing “Shut Down Guest” and “Restart Guest” all work as expected. If your experiences vary, please leave a comment

This should all be performed by the root user (otherwise prepend ‘sudo’ to the commands if you must use sudo).
The first thing you need to do is add the ‘contrib’ archive to your apt sources config:

nano /etc/apt/sources.list

Add the ‘contrib’ to the end of these source lines. I use the debian.uchicago.edu repository, so mine look like this on a new install:

deb http://debian.uchicago.edu/debian/ squeeze main contrib
deb-src http://debian.uchicago.edu/debian/ squeeze main contrib

After you save changes to the file, simply update your package list and perform the install:

apt-get update
apt-get install open-vm-tools open-vm-source

You might see an error that looks like this:

Loading open-vm-tools modules: vmhgfsFATAL: Module vmhgfs not found.
 vmmemctlFATAL: Module vmmemctl not found.
 vmsyncFATAL: Module vmsync not found.

Don’t worry, this is because we have not installed the kernel modules for some of the more advanced VMware functions like the vmhgfs file system, Memory Balloon (vmmemctl) Driver, and vmsync, which helps freeze the VM while taking snapshots.
To install these kernel modules, we’re going to use module-assistant. It will compile the modules for your kernel:

module-assistant auto-install open-vm -i

The ‘-i’ puts module assistant into non-interactive mode. It wont ask you to confirm any additional package installs necesarry to set up the build environment.
If you’re running a server without a Graphical environment, you’re finished. If you want features such as automatically adjusting the VM’s monitor resolution or allowing your mouse to seamlessly move between your host system and the VM, you need to install one more package:

apt-get install open-vm-toolbox

Now you won’t need to use CTRL+ALT to switch back and forth.

You can find out how much bandwidth your linux machine is using with a simple tool called “bwm-ng”. In Debian, install it with

aptitude install bwm-ng

Then, just type ‘bwm-ng’ in the command line. It will give you something like this:

bwm-ng v0.6 (probing every 5.000s), press 'h' for help
  input: /proc/net/dev type: rate
  -         iface                   Rx                   Tx                Total
  ==============================================================================
               lo:           0.00 KB/s            0.00 KB/s            0.00 KB/s
             eth0:        2221.47 KB/s           48.13 KB/s         2269.60 KB/s
  ------------------------------------------------------------------------------
            total:        2221.47 KB/s           48.13 KB/s         2269.60 KB/s

Pressing the “h” key while it is running wil actually pull up a nice menu to change some of the options you are looking at.

bwm-ng is very basic, “iptraf” is another tool that provides some more functionality if you want to drill further into what is moving in and out of your box.

If you install GNOME on Debian and intend to use it as a server, first thing you should do is remove the network-manager and network-manager-gnome packages. This tool can be helpful on a desktop, but since we are tweaking DNS, IP addresses, and perhaps even NIC bonding, the network-manager can screw up your settings.

aptitude purge network-manager-gnome network-manager 

If this is a VMware Virtual machine, install VMware tools. The prerequisites for Debian Lenny are:

aptitude install build-essential openssl libssl-dev linux-headers-$(uname -r) psmiscapt

You may have to manually create a DNS entry for your server. Just to be safe, do that now.

If you plan to support NTFS-like ACL’s, install support for it now.

aptitude install acl

now, you need to mount the partition with ACL’s enabled, to do this, edit your /etc/fstab.

nano /etc/fstab

The line I needed looked like this:

/dev/sda9 / ext3 acl,defaults 0 1

The important part is to get “acl” into the options list. Dont forget the comma.
I recommend rebooting just to make sure your fstab is set before you continue.
It’s easy to screw something up here and make your system unbootable, so if you reboot, and cant get your system back up and running, get into single user mode, and use this command. (You will have to adjust /dev/sda1 to your root partition. In my experience, it is usually /dev/sda1 or /dev/hda1. Use fdisk -l to list the partitions on the disk.

mount -o remount,rw /dev/sda1

Now lets get the samba and kerberos software packages installed.

aptitude install samba ntpdate smbclient winbind krb5-config krb5-user

Put in the Workgroup/domain info when prompted if you like. They will set up some .conf files for you, but we’re not going to use them anyways. We wont worry about WINS for now.
Now lets stop samba and winbind:

/etc/init.d/samba stop /etc/init.d/winbind stop

move the /etc/samba files somewhere

mkdir /etc/samba/vanillaconf mv /etc/samba/* /etc/samba/vanillaconf/

create your own /etc/samba/smb.conf: Make sure that the domain and realm fields are all caps.

[global] workgroup = ((DOMAIN)) server string = %h server wins support = no security = ads realm = ((DOMAIN)).COM encrypt passwords = yes obey pam restrictions = yes winbind use default domain = yes winbind enum users = yes winbind enum groups = yes template shell = /bin/bash idmap uid = 10000-20000 idmap gid = 10000-20000 

sync time

ntpdate ((domain controller))

now that samba knows what domain its part of, and we have a basic config for it, we need to get the authentication part working. Kerberos is the softare that communicates with your Active Directory domain to authenticate users.

nano /etc/krb5.conf

configure domain in /etc/krb5.conf. important parts:

[libdefaults] default_realm = ((domain)).COM [realms] ((DOMAIN.COM)) = { kdc = ((domain controller)) kdc = ((backup DC)) admin_server = (domain controller)) } [domain_realm] .(domain).com = (domain controller).(domain).COM

do a test by typing:

kinit [username]

then type:

klist

if you get some output looking like this, you are ready to go.

Ticket cache: FILE:/tmp/krb5cc_0 Default principal: (username)@(domain).COM Valid starting Expires Service principal 04/27/09 13:54:23 04/27/09 23:54:26 krbtgt/(domain).COM@(domain).COM renew until 04/27/09 23:54:23

To get Debian to recognize your Active Directory users, you need to update /etc/nsswitch.conf

nano /etc/nsswitch.conf

add “winbind” to the passwd and group lines. if you have “compat” in the line, put in like this:

passwd: compat winbind passwd_compat: winbind group: compat winbind group_compat: winbind 

Now start samba and winbind back up

/etc/init.d/winbind start /etc/init.d/samba start

Join your machine to the domain.

net ads join -U (administrative user)

You should see

Joined '(server name)' to realm '(domain).com'

If you see

No DNS domain configured for '(servername)'. Unable to perform DNS Update. DNS update failed!

Don’t worry. Just create an entry in your DNS server for the samba machine.
Test if winbind is working properly with

wbinfo -t

If you get

checking the trust secret via RPC calls failed Could not check secret

Then restart winbind:

/etc/init.d/winbind restart

allow AD accounts to logon to the machine:

# /etc/pam.d/common-account account sufficient pam_winbind.so account required pam_unix.so 

# /etc/pam.d/common-auth auth sufficient pam_winbind.so auth required pam_unix.so use_first_pass nullok_secure

# /etc/pam.d/common-session session required pam_mkhomedir.so skel=/etc/skel/ umask=0066 session sufficient pam_winbind.so session required pam_unix.so

It is critical that “obey pam restrictions” is set to “yes”. in your smb.conf for these pam settings to take effect.There are some other changes to these Pam.d settings i decided to make upon putting the server into production. I will write about that in another page [link].

Now, give your admin group from AD root access to the box

aptitude install sudo visudo

add this line:

%(domain)\\(admin's group) ALL =(ALL) ALL

When creating shares, to give an AD user access to a share, use:(More info in the future)

valid users = (domain)\username

to give an AD group access to a share, use:

valid users = @(domain)\groupname

Sometimes, I have to reboot my windows client machine to make it work… fyi.
Sources:

• O’reilly Samba book 3rd Edition.
• http://www.ccs.neu.edu/home/battista/documentation/winbind/index.html
• …Countless articles, mailing lists, forums.