Solaris Jumpstart - setting up the ISO

Step 1: Download the Solaris OS from Oracle
wget --http-user= --http-password= --no-check-certificate --output-document=sol-10-u9-ga-sparc-dvd-iso.zip http://download.oracle.com/otn/solaris/10/sol-10-u9-ga-sparc-dvd-iso.zip &

a. wget may be installed in /usr/sfw/bin, so check that location as well, before you go hunting for wget.
b. This command will seem like it's hanging, but tail the output file, and you'll see the download status.


Step2:
Unzip the file.

Step 3:
Mount the ISO
a. Create a block device for the ISO:

[root@gpadev-sol-x86:/export/depot]# lofiadm -a /export/depot/sol-10-u9-ga-sparc-dvd.iso
/dev/lofi/1

b. Mount it.
[root@gpadev-sol-x86:/export/depot]# mount -F hsfs /dev/lofi/1 /mnt

Step 3

a. Create a directory where you want the media to reside at.
mkdir -p /export/depot/s10upd9

b. Use the Solaris Tools to copy over the Install media to the directory you want. Dont just to a cp!

[root@gpadev-sol-x86:/export/depot/s10upd9]# cd /mnt/Solaris_10/Tools/
[root@gpadev-sol-x86:/mnt/Solaris_10/Tools]# ls
add_install_client Boot dial Installers rm_install_client setup_install_server

c. Use the "setup_install_server" command
[root@gpadev-sol-x86:/mnt/Solaris_10/Tools]# ./setup_install_server -w /export/depot/s10upd9 /export/depot/s10upd9
Verifying target directory...
Calculating the required disk space for the Solaris_10 product
Calculating space required for the installation boot image
Copying the CD image to disk...
Copying Install Boot Image hierarchy...
Copying /boot netboot hierarchy...
Install Server setup complete


Step 4:
Add to the Jumpstart server the location.

Step 5:
Remove the lofi device.

lofiadm -d /dev/lofi/1

Verify:
lofiadm

Sun Hardware handy tools

I came across ipmitool recently, and it is really handy when you need information about your machine (Serial Number, amount of memory etc)

Eg:
Whats the Serial Number of the Sun Blade?

ipmitool -H blade ILO IP-U root fru | grep "Product Serial"

Whats the Serial Number of the Sun Chassis
ipmitool -H chassis ilo ip -U root fru | grep "Product Serial"

How much memory do I have? (I had just installed some 4Gb dimms)
ipmitool -H blade ilo ip -U root fru | grep 4096MB | wc -l
Password:
24

i.e 24 dimms installed.

Solaris things...

I've been working with Solaris a lot, and we recently started with ZFS as well. Perhaps I'm missing something, but "top" & prstat dont report the correct amount of Memory free (RAM) on the machine when using zfs and neither does vmstat.

After some google searching, I found this helpful blog and thought I'd report. Thanks to "milek.blogspot.com" for this:

#!/bin/bash
#########
# Copied from http://milek.blogspot.com/2006/09/how-much-memory-does-zfs-consume.html
# to get correct reading of Free Memory
#

FREE=$(($(pagesize)*$(kstat -p unix:0:system_pages:freemem|awk '{print $NF}')))
ZIO_BUF=$(kstat -p zfs:0:arcstats:size|awk '{print $NF}')
FREE_MEM=$(((FREE+ZIO_BUF)/1024))
echo "Free Memory: $FREE_MEM MB"
FREE_MEM=$(((FREE+ZIO_BUF)/1024/1024))
echo "Free Memory: $FREE_MEM GB"

Next one was backups. We use Solaris 8 Branded Zones heavily to migrate off from our old Solaris environment. I started using ufsdump for backups, since this was not ZFS.

Well, trying to restore using ufsrestore, caused a ton of problems.

The simple

zoneadm -z s8zone install -a dumpfile

would give me weird errors such as " Sanity Check: Missing etc at /mnt"

Decided to use a ufsrestore, and that didnt help since zoneadm didn't know how to get it up.

Long story short, I did the following

1. Do a ufsrestore on to the disk
2. Used Zonecfg to create the configuration
3. Manually edited the /etc/zones/index file to modify the entry to match the backup version (with new name).

The index file would look like this:

global:installed:/
hostname:installed:/export/zones/app30qa:some id
hostname:installed:/export/zones/s8qa2flar:some id

This made zoneadm think that it was "installed" and I was finally able to boot up.

Sun Cluster 3.2 with LDOM 1.3

I had to get a Sun Cluster 3.2 to create a HA-NFS on LDOM 1.3. LDOM 1.3 and Sun Cluster 3.2 now fully support Sun Clusters on Guest Domains not just the I/O Domain. This makes it incredibly useful.

I faced plenty of issues, and hoping that this post will serve as a guide to those who need to do it.

Setup:

1. 2 Physical T-6340 blades.
2. Control Domain that also serves as our I/O Domain.
3. 1 Guest Domain on each Server for Sun Cluster.
3. EMC Powerpath installed on the Control Domain

Storage:

Managed by EMC Powerpath

The devices will be seen as : emcpowerXa

1 LUN for each Sun Cluster domain

1 Shared LUN between the 2 physical blades

I want to note STRONGLY that you should first try using the powerdevices for the LDOM & Sun Cluster, and only if that doesnt work use the cXtXdX devices. Each configuration may be different and you may have better luck or a patch that I did not have at the time of my trial.

powermt display dev= will give you the underlying cXtXdX number (in the I/O Domain).

Present all devices in MPGroups to the LDOM (even for boot disk since we'll be using SVM with Sun Cluster) as cXtXdX devices.

3 disks to be presented in this example

1. Boot Disk (mpgroup: bdskgroup)

2. Quorum Disk (mpgroup: quroumgroup)

3. Shared Disk for storage (mpgroup: cludskgroup)

(Make sure to give an id to each of the vdisks that are presented to the LDOM so that they always are presented in the same order even on a reboot)

For each Sun Cluster LDOM present the VNETs from the VSWs in the order they're cabled for the private interconnect. This is important.

I used cross over cables.

Jumpstart and boot up the LDOM.

If you've done everything correctly you should see 3 devices when you do "format".

Install the Sun Cluster software with the NFS packages. Patch it appropriately.

I used the below instructions to create the cluster, however, its far easier in 3.2. Just create the metadb by configuring the SVM (instructions in link below) and then run scinstall. Once thats done run clsetup and follow the instructions for the NFS Service.

http://napobo3.lk.net/sc/dual-node_HA_NFS_cluster_setup.html

Copied from the above link:
2 nodes (hostname: node1, node2) with installed Solaris 10 01/06, patches, Sun Cluster, NFS agent for Sun Cluster, VxFS
Both nodes are connected to the FC SAN storage, 8 storage LUNs are mapped to each node.

Configure SVM
On both nodes
Create the 25MB partition on the boot disk (s7)
Create the SVM database replica
metadb -afc 3 c0d0s7 (c0t0d0s7 on sparc)

On one node (node1)
- Create disk sets :
metaset -s nfs1 -a -h node1 node2
metaset -s nfs1 -t -f
metaset -s nfs1 -a /dev/did/rdsk/d2 /dev/did/rdsk/d3 /dev/did/rdsk/d4 /dev/did/rdsk/d5
metainit -s nfs1 d1 4 1 /dev/did/rdsk/d2s0 1 /dev/did/rdsk/d3s0 1 /dev/did/rdsk/d4s0 1 /dev/did/rdsk/d5s0
metastat -s nfs1 -p >> /etc/lvm/md.tab

metaset -s nfs2 -a -h node2 node1
metaset -s nfs2 -t -f
metaset -s nfs2 -a /dev/did/rdsk/d6 /dev/did/rdsk/d7 /dev/did/rdsk/d8 /dev/did/rdsk/d9
metainit -s nfs2 d1 4 1 /dev/did/rdsk/d6s0 1 /dev/did/rdsk/d7s0 1 /dev/did/rdsk/d8s0 1 /dev/did/rdsk/d9s0
metastat -s nfs2 -p >> /etc/lvm/md.tab
scp /etc/lvm/md.tab node2:/tmp/md.tab
ssh node2 'cat /tmp/md.tab >> /etc/lvm/md.tab'

- Create VxFS on shared devices -- I didnt do this.. I just did a newfs.
mkfs -F vxfs /dev/md/nfs1/rdsk/d1
mkfs -F vxfs /dev/md/nfs2/rdsk/d1

On both nodes
- Create the directories
mkdir -p /global/nfs1
mkdir -p /global/nfs2

http://docs.sun.com/app/docs/doc/819-0912/chapter1-1?l=en&a=view is the quick start guide, is some help as well (especially with the metadb setup), but they talk about LOCAL devices as HA-NFS, and that isn't want we're doing. :-)