IBM Spectrum Scale : Kerberized NFS with AD RFC2307

=== ITEMS COVERED ===

  1. Introduction
  2. Configure IBM Spectrum Scale for File AD RFC2307 NFS Kerberos authentication
    1. configure AD RFC2307 NFS Kerberos File Authentication
    2. nfs export creation and nfs config
  3. Configure RHEL client machine as AD+RFC2307+NFS_KERBEROS
    1. configure RHEL client machine as AD+RFC2307 client
    2. configure RHEL client machine as KERBEROS client (i.e. AD+RFC2307+KERBEROS at the end)
    3. nfs kerberos mount
    4. nfs kerberos access

=== 1. Introduction ===

This blog details the procedure to access data from IBM Spectrum Scale over kerberized NFS keeping Active Directory as the authentication server.

First, we explain how to configure IBM Spectrum Scale with File AD RFC2307 Authentication for NFS Kerberos access. Afterwords, the gory details of configuring RHEL machine as AD RFC2307 NFS Kerberos client.

Following is the summary of machines involved in NFS Kerberos access over AD RFC2307 :

  • active directory server : 10.0.100.27
  • active directory domain : ad.com
  • nfs client / RHEL client machine name : adclient1.ad.com
  • nfs server / spectrum scale netbios name : ckcluster.ad.com

=== 2. Configure IBM Spectrum Scale for File AD RFC2307 NFS Kerberos authentication ===

1. configure AD RFC2307 NFS Kerberos File Authentication

[root@cknode2 ] # mmuserauth service create –data-access-method file –type ad –servers 10.0.100.27 –user-name administrator –password Passw0rd –enable-nfs-kerberos –netbios-name ckcluster –idmap-role master –unixmap-domains “AD(320000001-320000090)”
File authentication configuration completed successfully.
[root@cknode2 ] #

[root@cknode2 ] # mmuserauth service list
FILE access configuration : AD
PARAMETERS VALUES
————————————————-
ENABLE_NFS_KERBEROS true
SERVERS 10.0.100.27
USER_NAME administrator
NETBIOS_NAME ckcluster
IDMAP_ROLE master
IDMAP_RANGE 10000000-299999999
IDMAP_RANGE_SIZE 1000000
UNIXMAP_DOMAINS AD(320000001-320000090)
LDAPMAP_DOMAINS none

OBJECT access not configured
PARAMETERS VALUES
————————————————-
[root@cknode2 ] #

 

2. nfs export creation and nfs config :

# command summary : create ganesha export for all security types + nfs protocols
mmcrfileset gpfs0 fset1
mmlinkfileset gpfs0 fset1 -J /ibm/gpfs0/fset1
chmod -R 777 /ibm/gpfs0/fset1
ls -ld /ibm/gpfs0/fset1
mmnfs export add /ibm/gpfs0/fset1 –client “*(Access_Type=RW,Squash=no_root_squash,SecType=sys:krb5:krb5i:krb5p)”
mmnfs export list -n /ibm/gpfs0/fset1
mmnfs config list

# command output :
[root@cknode2 ] # mmnfs export add /ibm/gpfs0/fset1 –client “*(Access_Type=RW,Squash=no_root_squash,SecType=sys:krb5:krb5i:krb5p)”
10.0.100.93: Redirecting to /bin/systemctl stop nfs-ganesha.service
10.0.100.92: Redirecting to /bin/systemctl stop nfs-ganesha.service
10.0.100.93: Redirecting to /bin/systemctl start nfs-ganesha.service
10.0.100.92: Redirecting to /bin/systemctl start nfs-ganesha.service
NFS Configuration successfully changed. NFS server restarted on all NFS nodes.
[root@cknode2 ] #
[root@cknode2 ] # mmnfs export list -n /ibm/gpfs0/fset1
Path Delegations Clients Access_Type Protocols Transports Squash Anonymous_uid Anonymous_gid SecType PrivilegedPort DefaultDelegations Manage_Gids NFS_Commit
————————————————————————————————————————————————————————————————-
/ibm/gpfs0/fset1 none * RW 3,4 TCP NO_ROOT_SQUASH -2 -2 SYS,KRB5,KRB5I,KRB5P FALSE none FALSE FALSE

[root@cknode2 ] #
[root@cknode2 ] # mmnfs config list

Idmapd Configuration
==========================
LOCAL-REALMS: AD.COM
DOMAIN: test.com
==========================
[root@cknode2 ] #


=== 3. Configure RHEL client machine as AD+RFC2307+NFS_KERBEROS ===

1. configure RHEL client machine as AD+RFC2307 client

# set FQDN hostname for your client
[root@adclient1 ~]# hostname
adclient1.ad.com
[root@adclient1 ~]#

# DNS Configuration (point to AD server)
[root@adclient1 ~]# cat /etc/resolv.conf
search ad.com
nameserver 10.0.100.27
[root@adclient1 ~]#

# INSTALL Pre req packages
[root@adclient1 ~]# yum install nfs-utils samba pam-krb5 samba-client.x86_64 samba-winbind.x86_64 samba-winbind-modules.x86_64 -y > /dev/null 2>&1
[root@adclient1 ~]#

# /etc/krb5.conf updates :
# set following in [libdefaults] section
default_realm = AD.COM
# set following in [realms] section
AD.COM = {
kdc = 10.0.100.27
}
# set following in [domain_realm] section
.ad.com = AD.COM
ad.com = AD.COM

# /etc/samba/smb.conf updates :
# set following entries in [global] section of /etc/samba/smb.conf file
workgroup = AD
realm = AD.COM
server string = Samba Server Version %v
security = ADS
password server = 10.0.100.27
log file = /var/log/samba/log.%m
max log size = 50
template shell = /bin/bash
idmap config AD:backend = ad
idmap config AD:range = 320000001-320000090
idmap config AD:schema_mode = rfc2307
idmap config * : range = 16777216-33554431
idmap config * : backend = tdb
cups options = raw
# you can see above changes using “testparm -s” command

# START winbind and smb services
[root@adclient1 ~]# service winbind restart
Redirecting to /bin/systemctl restart winbind.service
[root@adclient1 ~]# service smb restart
Redirecting to /bin/systemctl restart smb.service
[root@adclient1 ~]#

# Join the DOMAIN
[root@adclient1 ~]# net ads join -Uadministrator%Passw0rd
Using short domain name — AD
Joined ‘ADCLIENT3’ to dns domain ‘AD.COM’
[root@adclient1 ~]#
[root@adclient1 ~]# net ads testjoin
Join is OK
[root@adclient1 ~]#

# RESOLVE AD users on linux client
[root@adclient1 ~]# for i in `seq 1 5`; do id AD\\cruser$i; done
uid=320000001(AD\cruser1) gid=320000090(AD\crgroup1) groups=320000090(AD\crgroup1),4294967295,4294967295
uid=320000002(AD\cruser2) gid=320000090(AD\crgroup1) groups=320000090(AD\crgroup1),4294967295,4294967295
uid=320000003(AD\cruser3) gid=320000090(AD\crgroup1) groups=320000090(AD\crgroup1),4294967295,4294967295
uid=320000004(AD\cruser4) gid=320000090(AD\crgroup1) groups=320000090(AD\crgroup1),4294967295,4294967295
uid=320000005(AD\cruser5) gid=320000090(AD\crgroup1) groups=320000090(AD\crgroup1),4294967295,4294967295
[root@adclient1 ~]#
[root@adclient1 ~]# ##############
[root@adclient1 ~]# # linux machine configured as AD+RFC2307 client
[root@adclient1 ~]# ##############

 

2. configure RHEL client machine as KERBEROS client (i.e. AD+RFC2307+KERBEROS at the end)

# make sure all machines are time synchronized i.e. nfs client, AD server, nfs server

# FQDN Added to /etc/hosts for nfs client
[root@adclient1 ~]# grep ad.com /etc/hosts
10.0.100.216 ckcluster.ad.com ckcluster
10.0.100.217 ckcluster.ad.com ckcluster
10.0.100.251 adclient1.ad.com adclient1
[root@adclient1 ~]#

# CONFIGURE id mapping
# set Local-Realms and Domain attributes in /etc/idmapd.conf file
[root@adclient1 ~]# egrep “Local-Realms|Domain” /etc/idmapd.conf | grep -v ^#
Local-Realms = AD.COM
Domain = test.com
[root@adclient1 ~]#

# restart idmap service
[root@adclient1 ~]# service nfs-idmap start
Redirecting to /bin/systemctl start nfs-idmap.service
[root@adclient1 ~]#
[root@adclient1 ~]# ps -eaf | grep idmap
root 2786 1 0 17:02 ? 00:00:00 /usr/sbin/rpc.idmapd
root 2802 2721 0 17:02 pts/0 00:00:00 grep –color=auto idmap
[root@adclient1 ~]#

# CONFIGURE kerberos : create nfs kerberos keytab file for nfs client and copy it to nfs client
# To do this, execute following commands on AD server and scp/ftp keytab file to your client’s /etc/krb5.keytab path

# create AD user to map nfs principal for linux client
Users -> New User -> adclient1user-nfs -> adclient1user-nfs -> nfs/adclient1.ad.com@AD.COM -> Next -> Finish

# create nfs principal and keytab file for linux client
setspn -A nfs/adclient1.ad.com adclient1user-nfs
setspn -L adclient1user-nfs
ktpass -princ nfs/adclient1.ad.com@AD.COM -mapuser adclient1user-nfs -pass Passw0rd -crypto All -out adclient1.keytab

# command output given below
C:\Users\Administrator>
C:\Users\Administrator>setspn -A nfs/adclient1 adclient1user-nfs
Registering ServicePrincipalNames for CN=adclient1user-nfs,CN=Users,DC=AD,DC=COM
nfs/adclient1
Updated object

C:\Users\Administrator>setspn -A nfs/adclient1.ad.com adclient1user-nfs
Registering ServicePrincipalNames for CN=adclient1user-nfs,CN=Users,DC=AD,DC=COM
nfs/adclient1.ad.com
Updated object

C:\Users\Administrator>
C:\Users\Administrator>setspn -L adclient1user-nfs
Registered ServicePrincipalNames for CN=adclient1user-nfs,CN=Users,DC=AD,DC=COM:
nfs/adclient1.ad.com
nfs/adclient1

C:\Users\Administrator>
C:\Users\Administrator>ktpass -princ nfs/adclient1.ad.com@AD.COM -mapuser
adclient1user-nfs -pass Passw0rd -crypto All -out adclient1.keytab
Targeting domain controller: XXXAD.AD.COM
Successfully mapped nfs/adclient1.ad.com to adclient1user-nfs.
Password succesfully set!
WARNING: pType and account type do not match. This might cause problems.
Key created.
Key created.
Key created.
Key created.
Key created.
Output keytab to adclient1.keytab:
Keytab version: 0x502
keysize 60 nfs/adclient1.ad.com@AD.COM ptype 0 (KRB5_NT_UNKNOWN) vno 3 ety
pe 0x1 (DES-CBC-CRC) keylength 8 (0x76291c3e07d3d37c)
keysize 60 nfs/adclient1.ad.com@AD.COM ptype 0 (KRB5_NT_UNKNOWN) vno 3 ety
pe 0x3 (DES-CBC-MD5) keylength 8 (0x76291c3e07d3d37c)
keysize 68 nfs/adclient1.ad.com@AD.COM ptype 0 (KRB5_NT_UNKNOWN) vno 3 ety
pe 0x17 (RC4-HMAC) keylength 16 (0xa87f3a337d73085c45f9416be5787d86)
keysize 84 nfs/adclient1.ad.com@AD.COM ptype 0 (KRB5_NT_UNKNOWN) vno 3 ety
pe 0x12 (AES256-SHA1) keylength 32 (0xa6e1579382f43eac5c748ae93c09a16ad9d434505d
73c6f672589d4a7de57c9b)
keysize 68 nfs/adclient1.ad.com@AD.COM ptype 0 (KRB5_NT_UNKNOWN) vno 3 ety
pe 0x11 (AES128-SHA1) keylength 16 (0x86b5ea5c7fe237152148786e67b63c73)

C:\Users\Administrator>

# copy keytab file to linux client under /etc/krb5.keytab path
[root@adclient1 ~]#
[root@adclient1 ~]# klist -kte /etc/krb5.keytab
Keytab name: FILE:/etc/krb5.keytab
KVNO Timestamp Principal
—- —————– ——————————————————–
3 01/01/70 05:30:00 nfs/adclient1.ad.com@AD.COM (des-cbc-crc)
3 01/01/70 05:30:00 nfs/adclient1.ad.com@AD.COM (des-cbc-md5)
3 01/01/70 05:30:00 nfs/adclient1.ad.com@AD.COM (arcfour-hmac)
3 01/01/70 05:30:00 nfs/adclient1.ad.com@AD.COM (aes256-cts-hmac-sha1-96)
3 01/01/70 05:30:00 nfs/adclient1.ad.com@AD.COM (aes128-cts-hmac-sha1-96)
[root@adclient1 ~]#

# restart rpcgssd service
[root@adclient1 ~]# service nfs-secure start
Redirecting to /bin/systemctl start nfs-secure.service
[root@adclient1 ~]#
[root@adclient1 ~]# ps -eaf | grep gss
root 2800 1 0 17:02 ? 00:00:00 /usr/sbin/rpc.gssd
root 2804 2721 0 17:02 pts/0 00:00:00 grep –color=auto gss
[root@adclient1 ~]#
[root@adclient1 ~]#

 

3. nfs kerberos mount

# nfsv4 krb5i mount command
[root@adclient1 ~]# mount -t nfs -o vers=4,sec=krb5i ckcluster.ad.com:/ibm/gpfs0/fset1 /mnt/4k5i -vvvv
mount.nfs: timeout set for Tue Aug 11 17:47:42 2015
mount.nfs: trying text-based options ‘vers=4,sec=krb5i,addr=10.0.100.216,clientaddr=10.0.100.251’
[root@adclient1 ~]#

# client syslog entries for nfsv4 krb5i mount (if you are running rpcgssd in debug/verbose mode)
Aug 11 17:45:42 adclient1 rpc.gssd[9000]: handling gssd upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt6)
Aug 11 17:45:42 adclient1 rpc.gssd[9000]: handle_gssd_upcall: ‘mech=krb5 uid=0 enctypes=18,17,16,23,3,1,2 ‘
Aug 11 17:45:42 adclient1 rpc.gssd[9003]: handling krb5 upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt6)
Aug 11 17:45:42 adclient1 rpc.gssd[9003]: process_krb5_upcall: service is ‘<null>’
Aug 11 17:45:42 adclient1 rpc.gssd[9003]: Full hostname for ‘ckcluster.ad.com’ is ‘ckcluster.ad.com’
Aug 11 17:45:42 adclient1 rpc.gssd[9003]: Full hostname for ‘adclient1.ad.com’ is ‘adclient1.ad.com’
Aug 11 17:45:42 adclient1 rpc.gssd[9003]: No key table entry found for adclient1$@AD.COM while getting keytab entry for ‘adclient1$@AD.COM’
Aug 11 17:45:42 adclient1 rpc.gssd[9003]: No key table entry found for root/adclient1.ad.com@AD.COM while getting keytab entry for ‘root/adclient1.ad.com@AD.COM’
Aug 11 17:45:42 adclient1 rpc.gssd[9003]: Success getting keytab entry for ‘nfs/adclient1.ad.com@AD.COM’
Aug 11 17:45:42 adclient1 rpc.gssd[9003]: Successfully obtained machine credentials for principal ‘nfs/adclient1.ad.com@AD.COM’ stored in ccache ‘FILE:/tmp/krb5ccmachine_AD.COM’
Aug 11 17:45:42 adclient1 rpc.gssd[9003]: INFO: Credentials in CC ‘FILE:/tmp/krb5ccmachine_AD.COM’ are good until 1439331234
Aug 11 17:45:42 adclient1 rpc.gssd[9003]: using FILE:/tmp/krb5ccmachine_AD.COM as credentials cache for machine creds
Aug 11 17:45:42 adclient1 rpc.gssd[9003]: using environment variable to select krb5 ccache FILE:/tmp/krb5ccmachine_AD.COM
Aug 11 17:45:42 adclient1 rpc.gssd[9003]: creating tcp client for server ckcluster.ad.com
Aug 11 17:45:42 adclient1 rpc.gssd[9003]: DEBUG: port already set to 2049
Aug 11 17:45:42 adclient1 rpc.gssd[9003]: creating context with server nfs@ckcluster.ad.com
Aug 11 17:45:43 adclient1 rpc.gssd[9003]: DEBUG: serialize_krb5_ctx: lucid version!
Aug 11 17:45:43 adclient1 rpc.gssd[9003]: prepare_krb5_rfc4121_buffer: protocol 1
Aug 11 17:45:43 adclient1 rpc.gssd[9003]: prepare_krb5_rfc4121_buffer: serializing key with enctype 18 and size 32
Aug 11 17:45:43 adclient1 rpc.gssd[9003]: doing downcall lifetime_rec 35999
Aug 11 17:45:43 adclient1 rpc.gssd[9000]: destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt1c
Aug 11 17:45:43 adclient1 rpc.gssd[9000]: destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt1b

# nfsv4 krb5p mount command
[root@adclient1 ~]# mount -t nfs -o vers=4,sec=krb5p ckcluster.ad.com:/ibm/gpfs0/fset1 /mnt/4k5p -vvvv
mount.nfs: timeout set for Tue Aug 11 17:48:06 2015
mount.nfs: trying text-based options ‘vers=4,sec=krb5p,addr=10.0.100.216,clientaddr=10.0.100.251’
[root@adclient1 ~]#

# nfsv4 krb5 mount command
[root@adclient1 ~]# mount -t nfs -o vers=4,sec=krb5 ckcluster.ad.com:/ibm/gpfs0/fset1 /mnt/4k5 -vvvv
mount.nfs: timeout set for Tue Aug 11 17:48:19 2015
mount.nfs: trying text-based options ‘vers=4,sec=krb5,addr=10.0.100.216,clientaddr=10.0.100.251’
[root@adclient1 ~]#

 

4. nfs kerberos access

# AD user information on nfs client
[root@adclient1 ~]# id AD\\cruser2
uid=320000002(AD\cruser2) gid=320000090(AD\crgroup1) groups=320000090(AD\crgroup1),4294967295,16777217(BUILTIN\users)
[root@adclient1 ~]#

# krb5-workstation package provides kinit command which we use to create krb tickets
[root@adclient1 ~]# yum install krb5-workstation* -y >/dev/null 2>&1
[root@adclient1 ~]#

# create krb ticket for AD user and access nfs krb mounts
[root@adclient1 ~]# su AD\\cruser2
bash-4.2$ klist
klist: No credentials cache found (ticket cache KEYRING:persistent:320000002:320000002)
bash-4.2$
bash-4.2$ kinit -V -l 300s
Using default cache: persistent:320000002:320000002
Using principal: ADcruser2@AD.COM
kinit: Client not found in Kerberos database while getting initial credentials
bash-4.2$
bash-4.2$ kinit -V -l 300s cruser2
Using default cache: persistent:320000002:320000002
Using principal: cruser2@AD.COM
Password for cruser2@AD.COM:
Authenticated to Kerberos v5
bash-4.2$
bash-4.2$ klist
Ticket cache: KEYRING:persistent:320000002:320000002
Default principal: cruser2@AD.COM

Valid starting Expires Service principal
08/11/2015 17:54:26 08/11/2015 17:59:21 krbtgt/AD.COM@AD.COM
renew until 08/18/2015 17:54:21
bash-4.2$
bash-4.2$ date
Tue Aug 11 17:56:22 IST 2015
bash-4.2$

# nfs krb operations : access (cd), list (ls), create file (echo > file), read (cat), append (echo >> file)
bash-4.2$ cd /mnt/4k5
bash-4.2$ ls -ltr
bash-4.2$
bash-4.2$ echo ganesha > nnnk5
bash-4.2$ cat c2k5file1
bash-4.2$ echo adsflasdfsaoewroiwe >> nnnk5
bash-4.2$ cat nnnk5
ganesha
adsflasdfsaoewroiwe
bash-4.2$
bash-4.2$ cd /mnt/4k5i
bash-4.2$ cd /mnt/4k5p
bash-4.2$ cat nnnk5
ganesha
adsflasdfsaoewroiwe
bash-4.2$
bash-4.2$ >nnnk5
bash-4.2$
bash-4.2$ exit
[root@adclient1 ~]# pwd

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s