CPU Ready in esxi

Business Continuity Tiers (IBM, HP, Epic platforms)

Business Continuity Tier 1: Data backup with no hot site

Backup-Restore
Businesses that use Tier 1 Business Continuity solutions back up their data, typically with
tape, at an off-site facility. Depending on how often backups are made, they are prepared to
accept several days to weeks of data loss, but their backups are secure off-site. However,
this tier lacks the systems on which to restore data. Examples of Tier 1 Business Continuity
solutions include:

• Pickup Truck Access Method (PTAM), sending physical tapes (Vaulting company)
• Disk Subsystem or Tape based mirroring to locations without processors
• IBM Tivoli Storage Manager 7.x

Business Continuity Tier 2: Data backup with a hot site (current Tier)

Backup/Restore
Businesses using Tier 2 Business Continuity solutions make regular backups on tape. This
solution is combined with an off-site facility and infrastructure (known as a hot site) in which to
restore systems from those tapes in the event of a disaster. This tier of solution will still result
in the need to recreate several hours to days worth of data, but it is faster and more
consistent in response time when compared to Tier 1 Business Continuity solutions.
Examples of Tier 2 Business Continuity solutions include:

• PTAM with hot-site available (vaulting company & remote site)
• IBM Tivoli Storage Manager 7.x Disaster Recovery Manager (DRM)

Business Continuity Tier 3: Electronic vaulting

Backup/Restore
Tier 3 Business Continuity solutions use components of Tier 2 solutions. Additionally, some
mission critical data is electronically vaulted. This electronically vaulted data is typically more
current than that which is shipped with PTAM. These solutions also add higher levels of
automation, as a result there is less data recreation or loss. Examples of Tier 3 Business
Continuity solutions include:

• Electronic Vaulting of Data, (i.e. remote tape via channel extension, ProtecTier Replication)
• Remote site or remote data center
• IBM Tivoli Storage Manager 7.x3 (Disaster Recovery Manager)

Business Continuity Tier 4: Point-in-time copies (Desired Tier)

Rapid Data Recovery
Tier 4 Business Continuity solutions are used by businesses requiring both greater data
currency and faster recovery than users of lower tiers. Rather than relying largely on shipping
tape, as is common on the lower tiers, Tier 4 solutions begin to incorporate more disk based
solutions, typically using point in time disk copy. Several hours of data loss is still possible, but
it is easier to make such point-in-time (PIT) copies with greater frequency than data can be
replicated through tape based solutions. Examples of Tier 4 Business Continuity solutions
include:

• Data center to data center point-in-time copies
• HP Continuous Access
• HP Business Copy
• Peer-to-Peer Virtual Tape Server
• IBM Tivoli Storage Manager 7.x (Disaster Recovery Manager)
• IBM Tivoli Storage Manager for Copy Services
• IBM Tivoli Storage Manager for Advanced Copy Services
• TotalStorage® Productivity Center for Replication
• EMC Celerra IP Replication (Site 1 --> Site 2)

Business Continuity Tier 5: Transaction integrity (Desired Tier)

Rapid Data Recovery
Tier 5 Business Continuity solutions is a Business Continuity Tier reserved for application
software and database replication at the transaction level. This Business Continuity solution
are used by businesses with a requirement for consistency of data between production and
recovery data centers. There is little to no data loss in such solutions, however, the presence
of this functionality is entirely dependent on the applications in use. Examples of Tier 5
Business Continuity solutions include:

• Software, two-phase commit, such as DB2 remote replication, MQ Series
• Oracle Data Guard
• EPIC Caché Database Transaction Log Shipping

Business Continuity Tier 6: Zero or little data loss

Rapid Data Recovery
Tier 6 Business Continuity solutions maintain the highest levels of data currency. They are
used by businesses with little or no tolerance for data loss and who need to restore data to
applications rapidly. These solutions have no dependence on the applications to provide data
consistency, typically these solutions use real time storage mirroring or server mirroring.
Examples of Tier 6 Business Continuity solutions include:

• Metro Mirror
• Global Mirror
• z/OS Global Mirror
• GDPS HyperSwap™ Manager
• Peer-to-Peer VTS with synchronous write
• PPRC Migration Manager
• TotalStorage Productivity Center for Replication
• HACMP/XD with Logical Volume Mirroring

Business Continuity Tier 7: Highly automated, business integrated solution

Continuous Availability
Tier 7 Business Continuity solutions include all the major components being used for a Tier 6
Business Continuity solution with the additional integration of complete automation for
servers, storage, software and applications and network. This allows a Tier 7 Business
Continuity solution to ensure consistency of data above that which is granted by Tier 6
Business Continuity solutions. Additionally, recovery of the applications is automated,
allowing for restoration of systems and applications much faster and more reliably than would
be possible through manual Business Continuity procedures. Examples of Tier 7 Business
Continuity solutions include:

• GDPS/PPRC with or without HyperSwap
• GDPS/XRC
• GDPS/GM
• AIX® HACMP/XD with Metro Mirror
• System i High Availability Business Partner Software
• System i Copy Services Toolkit

System Profiles for CentOS/RedHat Linux

Operating at a large scale requires all systems to operate at the most optimized settings. A large-scale infrastructure has virtual hosts, virtual machines, and a wide variety of systems. Performance-related issues are a high percentage of trouble spots in an infrastructure. In a preemptive measure to reduce the number of issues arising from a system that is not properly tuned, configure tuned. tuned is a daemon in CentOS/RedHat that can modify system settings dynamically in conjunction with ktune.

Tuned

Tuned is a daemon that monitors the use of system components and dynamically tunes system settings based on that monitoring information. Dynamic tuning accounts for the way that various system components are used differently throughout the uptime for any given system. For example, the hard drive is used heavily during startup and login, but is barely used later when a user might mainly work with applications like OpenOffice or email clients. Similarly, the CPU and network devices are used differently at different times. Tuned monitors the activity of these components and reacts to changes in their use.

As a practical example, consider a typical office workstation. Most of the time, the Ethernet network interface will be very inactive. Only a few emails will go in and out every once in a while or some web pages might be loaded. For those kinds of loads, the network interface doesn't have to run at full speed all the time, as it does by default. Tuned has a monitoring and tuning plugin for network devices that can detect that low activity and then automatically lower the speed of that interface, typically resulting in lower power usage. If activity on the interface increases drastically for a longer period of time, for example because a DVD image is being downloaded or an email with a large attachment is opened, tuned detects this and sets the interface speed to maximum to offer the best performance while the activity level is so high. This principle is used for the other plugins for CPU and hard disks as well.

Network devices are not configured to behave this way by default because speed changes can take several seconds to take effect and therefore directly and visibly impact the user experience. Similar considerations apply for the CPU and hard drive tuning plugins. When a hard drive has been spun down, it can take several seconds for it to spin up again which results in an observed lack of responsiveness of the system during that period. The latency side effect is smallest for the CPU plugin, but it is still at least measurable, though hardly noticeable by a user.

How to install tuned

# yum install tuned

Loaded plugins: fastestmirror, refresh-packagekit, security
Setting up Install Process
Loading mirror speeds from cached hostfile
epel/metalink                                                                                                                         |  14 kB     00:00
 * base: ftpmirror.your.org
 * epel: mirror.steadfast.net
 * extras: mirrors.easynews.com
 * updates: linux.cc.lehigh.edu
base                                                                                                                                  | 3.7 kB     00:00
epel                                                                                                                                  | 4.4 kB     00:00
epel/primary_db                                                                                                                       | 6.4 MB     00:13
extras                                                                                                                                | 3.4 kB     00:00
isv_ownCloud_community                                                                                                                | 1.2 kB     00:00
updates                                                                                                                               | 3.4 kB     00:00
updates/primary_db                                                                                                                    | 1.5 MB     00:01
Resolving Dependencies
--> Running transaction check
---> Package tuned.noarch 0:0.2.19-13.el6_6.1 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=============================================================================================================================================================
 Package                          Arch                              Version                                         Repository                          Size
=============================================================================================================================================================
Installing:
 tuned                            noarch                            0.2.19-13.el6_6.1                               updates                             94 k

Transaction Summary
==================================================================================================================================================
Install       1 Package(s)

Total download size: 94 k
Installed size: 219 k
Is this ok [y/N]: y
Downloading Packages:
tuned-0.2.19-13.el6_6.1.noarch.rpm                                                                                                    |  94 kB
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : tuned-0.2.19-13.el6_6.1.noarch
  Verifying  : tuned-0.2.19-13.el6_6.1.noarch

Installed:
  tuned.noarch 0:0.2.19-13.el6_6.1

How to interact with tuned

# tuned-adm list
Available profiles:
- server-powersave
- enterprise-storage
- default
- desktop-powersave
- laptop-ac-powersave
- virtual-guest
- laptop-battery-powersave
- throughput-performance
- latency-performance
- virtual-host
- spindown-disk

Next, view which profile is active. In my case, a default profile is active.

# tuned-adm active
Current active profile: default
Service tuned: disabled, stopped
Service ktune: disabled, stopped

I will now change the active profile to throughput-performance

# tuned-adm profile throughput-performance
Switching to profile 'throughput-performance'
Applying deadline elevator: dm-0 dm-1 dm-2 sda sdb         [  OK  ]
Applying ktune sysctl settings:
/etc/ktune.d/tunedadm.conf:                                [  OK  ]
Calling '/etc/ktune.d/tunedadm.sh start':                  [  OK  ]
Applying sysctl settings from /etc/sysctl.conf
Starting tuned:                                            [  OK  ]

# tuned-adm active
Current active profile: throughput-performance
Service tuned: enabled, running
Service ktune: enabled, running

When you first install tuned, the default profile will be active. Red Hat Enterprise Linux 6 also includes the following predefined profiles:

default
The default power-saving profile. It has the lowest impact on power saving of the available profiles and only enables CPU and disk plugins of tuned.
desktop-powersave
A power-saving profile directed at desktop systems. Enables ALPM power saving for SATA host adapters as well as the CPU, Ethernet, and disk plugins of tuned.
server-powersave
A power-saving profile directed at server systems. Enables ALPM powersaving for SATA host adapters, disables CD-ROM polling through HAL (refer to the hal-disable-polling man page) and activates the CPU and disk plugins of tuned.
laptop-ac-powersave
A medium-impact power-saving profile directed at laptops running on AC. Enables ALPM powersaving for SATA host adapters, WiFi power saving, as well as the CPU, Ethernet and disk plugins of tuned.
laptop-battery-powersave
A high-impact power-saving profile directed at laptops running on battery. It activates all power saving mechanisms from the previous profiles plus it enables the multi-core power-savings scheduler for low wakeup systems and makes sure that the ondemand governor is active and that AC97 audio power-saving is enabled. You can use this profile to save the maximum amount of power on any kind of system, not only laptops on battery power. The tradeoff for this profile is a noticeable impact on performance, specifically latency of disk and network I/O.
spindown-disk
A strong power-saving profile directed at machines with classic hard disks. It enables aggressive disk spin-down by increasing disk writeback values, lowering disk swappiness, and disabling log syncing. All partitions are remounted with a noatime option. All tuned plugins are disabled.
throughput-performance
A server profile for typical throughput performance tuning. It disables tuned and ktune power saving mechanisms, enables sysctl settings that improve the throughput performance of your disk and network I/O, and switches to the deadline scheduler.
latency-performance
A server profile for typical latency performance tuning. This profile disables dynamic tuning mechanisms and transparent hugepages. It uses the performance governer for p-states through cpuspeed, and sets the I/O scheduler to deadline. Additionally, in Red Hat Enterprise Linux 6.5 and later, the profile requests a cpu_dma_latency value of 1. In Red Hat Enterprise Linux 6.4 and earlier, cpu_dma_latency requested a value of 0.
enterprise-storage
A server profile to improve throughput performance for enterprise-sized server configurations. This switches to the deadline scheduler and disables certain I/O barriers, dramatically improving throughput.
virtual-guest
This profile is optimized for virtual machines. It is based on the enterprise-storage profile, but also decreases the swappiness of virtual memory. This profile is available in Red Hat Enterprise Linux 6.3 and later.
virtual-host
Based on the enterprise-storage profile, virtual-host decreases the swappiness of virtual memory and enables more aggressive writeback of dirty pages. Non-root and non-boot file systems are mounted with barrier=0. Additionally, as of Red Hat Enterprise Linux 6.5, the kernel.sched_migration_cost parameter is set to 5 milliseconds. Prior to Red Hat Enterprise Linux 6.5, kernel.sched_migration_cost used the default value of 0.5 milliseconds.
sap
A profile optimized for the best performance of SAP software. It is based on the enterprise-storage profile. The sap profile additionally tunes the sysctl settings regarding shared memory, semaphores and the maximum number of memory map areas a process may have.


If you want to view all the different settings that have been changed for a given profile, you can do so by looking at the /etc/tune-profiles/ directory, which lists each profile and the settings that are changed based on the profile.

START/STOP IBM Tivoli Storage Manager Administration Center

IBM Tivoli Storage Manager (TSM) is a centralized, policy-based, enterprise class, data backup and recovery package. The software enables the user to insert objects not only via backup, but also through space management and archive tools. It also allows retrieval of the same data via similar restore, recall, and retrieve methods.


START

# cd/opt/IBM/tivoli/tipv21/tipv2Components/TCRComponent/bin
# ./startTCRserver.sh

Using /opt/IBM/tivoli/tipv21/tipv2/java/jre/bin/java
ADMU0116I: Tool information is being logged in file
           /opt/IBM/tivoli/tipv21/tipv2/profiles/TIPProfile/logs/server1/startServer.log
ADMU0128I: Starting tool with the TIPProfile profile
ADMU3100I: Reading configuration for server: server1
ADMU3200I: Server launched. Waiting for initialization status.
ADMU3000I: Server server1 open for e-business; process id is 7209150

STOP

# cd /opt/IBM/tivoli/tipv21/tipv2Components/TCRComponent/bin
# ./stopTCRserver.sh

Using /opt/IBM/tivoli/tipv21/tipv2/java/jre/bin/java
ADMU0116I: Tool information is being logged in file
           /opt/IBM/tivoli/tipv21/tipv2/profiles/TIPProfile/logs/server1/stopServer.log
ADMU0128I: Starting tool with the TIPProfile profile
ADMU3100I: Reading configuration for server: server1
Realm/Cell Name:
Username: tipadmin
Password: ********                                                                                                                                             ADMU3201I: Server stop request issued. Waiting for stop status.
ADMU4000I: Server server1 stop completed.

Working with Snapshots using VMware vSphere PowerCLI

Create Snapshot
Creates a new snapshot of a virtual machine with the provided inputs.

Syntax:
New-Snapshot [-Name] [-Description ] [-Memory] [-Quiesce] [-RunAsync] [-VM] [-Server ] [-WhatIf] [-Confirm] []
Example:
New-Snapshot -VM MYSERVER -name MYSERVERBEFORECHROME
Name                     Description                  PowerState
----                     -----------                  ----------
MYSERVERBEFORECHROME                                  PoweredOff

Remove Snapshot
Removes the specified virtual machine snapshots.

Syntax:
Remove-Snapshot [-Snapshot] [-RemoveChildren] [-RunAsync] [-WhatIf] [-Confirm] []
Example:
Remove-Snapshot -Snapshot MYSERVERBEFORECHROME -RemoveChildren

List Snapshot

get-vm | get-snapshot | format-list
get-vm | get-snapshot | format-list vm,name

Delete Snapshot

$snap=get-snapshot -vm MYSERVER; remove-snapshot -snapshot $snapChildren] [-RunAsync] [-WhatIf] [-Confirm] []

Resolving .local domain names in Linux Mint 17.x and Ubuntu 14.0x

The good old days of manually editing /etc/resolv.conf and network configuration files are gone. If you are running latest versions of Ubuntu and Linux Mint the Avahi daemon intercepts all .local DNS requests and does something with it, except resolving it to the correct host. Lovely.

Here is the solution to this pesky "feature":

1. Edit /etc/nsswitch.conf

2. Change the hosts line to hosts: files dns mdns4_minimal [NOTFOUND=return] mdns4 (move dns from the end)

MINTY jblanco # vim  /etc/nsswitch.conf
# /etc/nsswitch.conf
#
# Example configuration of GNU Name Service Switch functionality.
# If you have the `glibc-doc-reference' and `info' packages installed, try:
# `info libc "Name Service Switch"' for information about this file.

passwd:         compat
group:          compat
shadow:         compat

hosts:          files dns mdns4_minimal [NOTFOUND=return]
networks:       files

protocols:      db files
services:       db files
ethers:         db files
rpc:            db files

netgroup:       nis
~


Avahi is a system which facilitates service discovery on a local network via the mDNS/DNS-SD protocol suite. This enables you to plug your laptop or computer into a network and instantly be able to view other people who you can chat with, find printers to print to or find files being shared. 

Configuring TSM Client Acceptor Daemon under Ubuntu 12.04

#!/bin/bash
DESC="TSM Client Accpetor Daemon"
NAME="dsmcad"
DAEMON=/opt/tivoli/tsm/client/ba/bin/$NAME
SCRIPTNAME="/etc/init.d/$NAME"

. /lib/lsb/init-functions

DSM_CONFIG=/opt/tivoli/tsm/client/ba/bin/dsm.opt
DSM_DIR=/opt/tivoli/tsm/client/ba/bin/
DSM_LOG=/opt/tivoli/tsm/client/ba/bin/
LANG=en_US.utf8
LC_ALL=en_US.utf8

export DSM_DIR DSM_CONFIG DSM_LOG LANG LC_ALL

rootcheck() {
if [ $UID -ne 0 ] ; then
    echo "You must be a root user" 2>&1
        exit 1
fi
}
pidof_dsmcad() {
PID=$(pidof $DAEMON)
if [ ! $PID ]; then
    return 1
fi
echo $PID
}
d_start () {
    rootcheck
    if [ ! $(pidof_dsmcad) ]; then
            $DAEMON > /dev/null 2>&1
    fi
}
d_stop() {
    rootcheck
    pkill -f $DAEMON
}
d_status() {
    if [ $(pidof_dsmcad) ]; then
               echo "$DESC is running"
    else
        echo "$DESC is not running"
fi
}

case $1 in
    start)
        log_daemon_msg "Starting" $DESC $DAEMON
        d_start
        if [ $(pidof dsmcad) ]; then
            log_end_msg 0
        else
            log_end_msg 1
        fi
;;
    stop)
        log_daemon_msg "Stopping" $DESC $DAEMON
        d_stop
        sleep 2
        if [ $(pidof $DAEMON) ]; then
            log_end_msg 1
        else
            log_end_msg 0
        fi
;;
    status)
               d_status
;;
    restart)
            $0 stop
        sleep 1
        $0 start
;;

    *)
        echo "Usage: /etc/init.d/dsmcad {start|stop|restart|status}"
;;
esac

Installing the TSM client on an Ubuntu machine

Target OS: Ubuntu 12.04 Desktop
TSM Client Version: 6.4.0.0


sudo apt-get install ksh libstdc++5 alien

ftp://service.boulder.ibm.com/storage/tivoli-storage-management/maintenance/client/v6r4/Linux/LinuxX86/BA/v640/6.4.0.0-TIV-TSMBAC-LinuxX86.tar

# mkdir 6.4.0.0-TIV-TSMBAC-LinuxX86.ORIG
# mkdir 6.4.0.0-TIV-TSMBAC-LinuxX86
# cp 6.4.0.0-TIV-TSMBAC-LinuxX86.ORIG/6.4.0.0-TIV-TSMBAC-LinuxX86.tar 6.4.0.0-TIV-TSMBAC-LinuxX86
# cd 6.4.0.0-TIV-TSMBAC-LinuxX86
# tar xvf 6.4.0.0-TIV-TSMBAC-LinuxX86.tar

# alien -c -g TIVsm-API64.x86_64.rpm
# alien -c -g TIVsm-BA.x86_64.rpm
# alien -c -g gskcrypt64-8.0.14.11.linux.x86_64.rpm
# alien -c -g gskssl64-8.0.14.11.linux.x86_64.rpm

# mv TIVsm-API64-6.4.0/debian TIVsm-API64-6.4.0/DEBIAN
# mv TIVsm-BA-6.4.0/debian TIVsm-BA-6.4.0/DEBIAN
# mv gskssl64-8.0/debian gskssl64-8.0/DEBIAN
# mv gskcrypt64-8.0/debian gskcrypt64-8.0/DEBIAN

edit control packages


    remove blank line after "Maintainer:" line
    in "Architecture:" line change "i386" to "amd64"
    delete "${shlibs:Depends}" in "Depends:" line
    joing two "Description:" lines into one line
    insert line "Version: 6.4.0"
    delete last two lines " ." and " (Converted..."
    append an empty line as last line


# dpkg -b TIVsm-API64-6.4.0
# dpkg -b TIVsm-BA-6.4.0
# dpkg -b gskssl64-8.0
# dpkg -b gskcrypt64-8.0

chmod 0775 postinst and other scripts

# dpkg -i TIVsm-API64-6.4.0.deb
# dpkg -i TIVsm-BA-6.4.0.deb
# dpkg -i gskssl64-8.0.deb
# dpkg -i gskcrypt64-8.0.deb


Add the TSM libraries to your linker config path, and rescan.

# echo "/opt/tivoli/tsm/client/api/bin" > /etc/ld.so.conf.d/tsm.conf
# echo "/opt/tivoli/tsm/client/api/bin64" >> /etc/ld.so.conf.d/tsm.conf
# echo "/usr/local/ibm/gsk8_64/lib64" >> /etc/ld.so.conf.d/tsm.conf
# echo "/usr/lib64" >> /etc/ld.so.conf.d/tsm.conf

# ldconfig

Make a link so the client knows where to find the language catalogs

# ln -s /opt/tivoli/tsm/client/lang/EN_US /opt/tivoli/tsm/client/ba/bin/EN_US

Ubuntu and Proxy Servers

apt-get, aptitude, etc. will not obey the environment variables when used normally with sudo. So separately configure them; create a file called apt.conf in /etc/apt and include the following:

Acquire::http::proxy "http://myproxy.server.com:8080/";
Acquire::ftp::proxy "ftp://myproxy.server.com:8080/";
Acquire::https::proxy "https://myproxy.server.com:8080/";

Some web proxy servers will require something like this:

Acquire::http::proxy "http://DOMAIN\username:passwd@proxy.server.com:port/";

A script to show number of mounted tapes

#!/bin/sh
#Set Variables
TSM_CMD="dsmadmc -id=admin -password=******** -dataonly=yes"

#Main
TAPE_MOUNTS=`${TSM_CMD} q mo`
#echo $TAPE_MOUNTS
TAPE_DRIVE_COUNT=0
for item in $TAPE_MOUNTS; do
#echo $(echo $item | egrep 3592DRIVE[01,02,03,04,05,06,07,08,09,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24] )
TAPE_DRIVE=$(echo $item | egrep 3592DRIVE[01,02,03,04,05,06,07,08,09,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24] )
        if [ ! -z $TAPE_DRIVE ]; then
                ((TAPE_DRIVE_COUNT=TAPE_DRIVE_COUNT +1))
        fi
done
echo "Total 3592 Mounted Tapes=" $TAPE_DRIVE_COUNT

A script to check your WWPNs

#!/bin/ksh
#Author : Javier Blanco
#Location : Hollywood, FL
#Contacts :
#Phone :
######### Subject ######### # # #
Check the Fiber Adapters and its WWPN # # # #############################################
lsdev -Cc adapter|grep fcs|awk '{print $1,$3}'|while read fcs slot
do
wwpn=`lscfg -vl $fcs | grep -i network | cut -c 37-60`
echo $fcs $slot $wwpn
done

Displays all lines not matching the specified pattern

grep -v '^*' dsmserv.opt

How to move those nasty MOV files to an external disk

I was trying to create a new virtual machine using VMware Fusion and it turns out I am out space on my 500GB hard drive. What? 10GB left?

From Terminal:

#cd /Users/jabschmidt
#du -sh * Shows I have 330GB worth of Pictures/Movies.

I know MOVs are bigger in size than jpg pictures so I decided to move all of my MOVs to my external drive.

#pwd /Users/jabschmidt/Pictures/2012
#find . -name \*.MOV -exec echo mv -i {} "/Volumes/My Passport/MOVS/2012/" \;

mv -i Cause mv to write a prompt to standard error before moving a file that would overwrite an existing file. If the response from the standard input begins with the character `y' or `Y', the move is attempted.

#find . -name \*.MOV -exec echo mv -i {} "/Volumes/My Passport/MOVS/2011/" \;
#find . -name \*.MOV -exec echo mv -i {} "/Volumes/My Passport/MOVS/2012/" \;
#find . -name \*.MOV -exec echo mv -i {} "/Volumes/My Passport/MOVS/2013/" \;

 If you find duplicate MOV names, do another pass using a different folder name, i.e:

#find . -name \*.MOV -exec echo mv -i {} "/Volumes/My Passport/MOVS/2011/MOREMOVS/" \;

VBScript :: Ping Servers and Export To Microsoft Excel

This is a useful VBScript I always keep around. It requires Microsoft Excel (tested with 2003/2007), a text file with a list of host names (Windows servers) and the Command Line to run

The file servers.txt must look like this:

HOSTNAME1
HOSTNAME2
HOSTNAME3
.
.
.
HOSTNAMEn

This is the syntax to execute it:

C:\>cscript ping_servers.vbs

AIX's kill command

Many people ask me which is the best practice for the kill command. My preferred SignalName is -TERM or -15. This signal can be trapped.

kill [  - SignalName |  - SignalNumber ] ProcessID ...

The kill command sends a signal (by default, the SIGTERM signal) to a running process. This default action normally stops processes. If you want to stop a process, specify the process ID (PID) in the ProcessID variable. The shell reports the PID of each process that is running in the background (unless you start more than one process in a pipeline, in which case the shell reports the number of the last process). You can also use the ps command to find the process ID number of commands.

A root user can stop any process with the kill command. If you are not a root user, you must have initiated the process you want to stop.

SignalName is recognized in a case-independent fashion, without the SIG prefix.

If the specified SignalNumber is 0, the kill command checks the validity of the specified PID.

• -1 is the most polite.  It is a gentleman's agreement between Admin & Programmer.
• -15 is from AIX.  It's the thing that happens when 'shutdown' says "All processes currently running will now be killed..."
• -9, obviously, is low-down-dirty & mean.  The process doesn't get the chance to clean up at all.  Not ideal...

-HUP (1)
-TERM (15)
-KILL (9)




MapReduce

MapReduce is a framework introduced by Google for processing larges amounts of data.

The framework uses a simple idea derived from the commonly known map and reduce functions used in functional programming (ex: LISP). It divides the main problem into smaller sub-problems and distribute these to a cluster of computers. It then combines the answers to these sub-problems to obtain a final answer.

MapReduce facilitates the process of distributed computing making possible that users with no knowledge on the subject create their own distributed applications. The framework hides all the details of parallelization, data distribution load balancing and fault tolerance and the user basically has only to specify the Map and the Reduce functions.

In the process, the input is divided into small independent chunks. The map function receives a piece of the input, processes it, and passes the input in the format key/value pair as answer. These key/values are grouped in a certain way and given as input to the reduce function. This in its turn merges the values, giving the final answer.

Each map and each reduce may be processed by a different node (Computer) in the cluster. The quantity of nodes in the cluster may be as big as the availability of computers in your network. The framework is responsible for dividing the input and feeding the map function. Afterwards, it collects  map's outputs, group and send them to the reduce function. After the work of reduce if done, the framework gather the answers in a final output.

The following picture shows the MapReduce flow

Sucessful Import from Wordpress

I successfully imported my old posts from Wordpress. First I had to EXPORT my blog from Wordpress (Site Admin | Tools | Available Tools | Export).

Then uploaded the XML file to WordPress2Blogger online service to make it Blogger compatible.

Finally, I used the IMPORT function from Blooger. Voilà!

Ubuntu - Nagios Installation

This guide is intended to provide you with simple instructions on how to install Nagios from source (code) on Ubuntu and have it monitoring your local machine inside of 20 minutes. No advanced installation options are discussed here - just the basics that will work for 95% of users who want to get started.

These instructions were written based on an Ubuntu 6.10 (desktop) installation. They should work for an Ubuntu 7.10 install as well. I used a virtual machine running Ubuntu JeOS back in 2009.

If you follow these instructions, here's what you'll end up with:

Nagios and the plugins will be installed underneath /usr/local/nagios
Nagios will be configured to monitor a few aspects of your local system (CPU load, disk usage, etc.)
The Nagios web interface will be accessible at http://localhost/nagios/

Required Packages

Make sure you've installed the following packages on your Ubuntu installation before continuing.

Apache 2
GCC compiler and development libraries
GD development libraries
You can use apt-get to install these packages by running the following commands:

sudo apt-get install apache2
sudo apt-get install build-essential

With Ubuntu 6.10, install the gd2 library with this command:

sudo apt-get install libgd2-dev

With Ubuntu 7.10, the gd2 library name has changed, so you'll need to use the following:

sudo apt-get install libgd2-xpm-dev

1) Create Account Information

Become the root user.

sudo -s

Create a new nagios user account and give it a password.

/usr/sbin/useradd nagios
passwd nagios

On Ubuntu server edition (6.01 and possible newer versions), you will need to also add a nagios group (it's not created by default). You should be able to skip this step on desktop editions of Ubuntu.

/usr/sbin/groupadd nagios
/usr/sbin/usermod -G nagios nagios

Create a new nagcmd group for allowing external commands to be submitted through the web interface. Add both the nagios user and the apache user to the group.

/usr/sbin/groupadd nagcmd
/usr/sbin/usermod -G nagcmd nagios
/usr/sbin/usermod -G nagcmd www-data

2) Download Nagios and the Plugins

Create a directory for storing the downloads.

mkdir ~/downloads
cd ~/downloads

Download the source code tarballs of both Nagios and the Nagios plugins (visit http://www.nagios.org/download/ for links to the latest versions). At the time of writing, the latest versions of Nagios and the Nagios plugins were 3.0 and 1.4.11, respectively.


wget http://prdownloads.sourceforge.net/nagios/nagios-3.0.1.tar.gz
wget http://osdn.dl.sourceforge.net/sourceforge/nagiosplug/nagios-plugins-1.4.11.tar.gz


3) Compile and Install Nagios

Extract the Nagios source code tarball.

cd ~/downloads
tar xzf nagios-3.0.tar.gz
cd nagios-3.0

Run the Nagios configure script, passing the name of the group you created earlier like so:

./configure --with-command-group=nagcmd

Compile the Nagios source code.

make all

Install binaries, init script, sample config files and set permissions on the external command directory.

make install
make install-init
make install-config
make install-commandmode

Don't start Nagios yet - there's still more that needs to be done...

4) Customize Configuration

Sample configuration files have now been installed in the /usr/local/nagios/etc directory. These sample files should work fine for getting started with Nagios. You'll need to make just one change before you proceed...

Edit the /usr/local/nagios/etc/objects/contacts.cfg config file with your favorite editor and change the email address associated with the nagiosadmin contact definition to the address you'd like to use for receiving alerts.

vi /usr/local/nagios/etc/objects/contacts.cfg

5) Configure the Web Interface

Install the Nagios web config file in the Apache conf.d directory.

make install-webconf

Create a nagiosadmin account for logging into the Nagios web interface. Remember the password you assign to this account - you'll need it later.

htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin

Restart Apache to make the new settings take effect.

/etc/init.d/apache2 reload

6) Compile and Install the Nagios Plugins

Extract the Nagios plugins source code tarball.

cd ~/downloads
tar xzf nagios-plugins-1.4.11.tar.gz
cd nagios-plugins-1.4.11

Compile and install the plugins.

./configure --with-nagios-user=nagios --with-nagios-group=nagios
make
make install

7) Start Nagios

Configure Nagios to automatically start when the system boots.

ln -s /etc/init.d/nagios /etc/rcS.d/S99nagios

Verify the sample Nagios configuration files.

/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

If there are no errors, start Nagios.

/etc/init.d/nagios start

8) Login to the Web Interface

You should now be able to access the Nagios web interface at the URL below. You'll be prompted for the username (nagiosadmin) and password you specified earlier.

http://localhost/nagios/

Click on the "Service Detail" navbar link to see details of what's being monitored on your local machine. It will take a few minutes for Nagios to check all the services associated with your machine, as the checks are spread out over time.

9) Other Modifications

If you want to receive email notifications for Nagios alerts, you need to install the mailx (Postfix) package.

sudo apt-get install mailx

You'll have to edit the Nagios email notification commands found in /usr/local/nagios/etc/objects/commands.cfg and change any '/bin/mail' references to '/usr/bin/mail'. Once you do that you'll need to restart Nagios to make the configuration changes live.

sudo /etc/init.d/nagios restart

Configuring email notifications is outside the scope of this documentation. Refer to your system documentation, search the web, or look to the NagiosCommunity.org wiki for specific instructions on configuring your Ubuntu system to send email messages to external addresses.

Ubuntu 101

From old Yahoo Notes I found this good intro to Ubuntu VMs.

Logs: A "live" view of a logfile on Linux

tail -f /path/thefile.log

Storage: Adding Hard drive

jblanco@us01:~$ sudo fdisk -l

Disk /dev/sda: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 1 1247 10016496 83 Linux
/dev/sda2 1248 1305 465885 5 Extended
/dev/sda5 1248 1305 465853+ 82 Linux swap / Solaris

Disk /dev/sdb: 42.9 GB, 42949672960 bytes
255 heads, 63 sectors/track, 5221 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdb doesn't contain a valid partition table
jblanco@us01:~$ fdisk /dev/sdb

Unable to open /dev/sdb
jblanco@us01:~$ sudo fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.

The number of cylinders for this disk is set to 5221.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): m
Command action
a toggle a bootable flag
b edit bsd disklabel
c toggle the dos compatibility flag
d delete a partition
l list known partition types
m print this menu
n add a new partition
o create a new empty DOS partition table
p print the partition table
q quit without saving changes
s create a new empty Sun disklabel
t change a partition's system id
u change display/entry units
v verify the partition table
w write table to disk and exit
x extra functionality (experts only)

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-5221, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-5221, default 5221):
Using default value 5221

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): L

0 Empty 1e Hidden W95 FAT1 80 Old Minix be Solaris boot
1 FAT12 24 NEC DOS 81 Minix / old Lin bf Solaris
2 XENIX root 39 Plan 9 82 Linux swap / So c1 DRDOS/sec (FAT-
3 XENIX usr 3c PartitionMagic 83 Linux c4 DRDOS/sec (FAT-
4 FAT16 <32M 40 Venix 80286 84 OS/2 hidden C: c6 DRDOS/sec (FAT-
5 Extended 41 PPC PReP Boot 85 Linux extended c7 Syrinx
6 FAT16 42 SFS 86 NTFS volume set da Non-FS data
7 HPFS/NTFS 4d QNX4.x 87 NTFS volume set db CP/M / CTOS / .
8 AIX 4e QNX4.x 2nd part 88 Linux plaintext de Dell Utility
9 AIX bootable 4f QNX4.x 3rd part 8e Linux LVM df BootIt
a OS/2 Boot Manag 50 OnTrack DM 93 Amoeba e1 DOS access
b W95 FAT32 51 OnTrack DM6 Aux 94 Amoeba BBT e3 DOS R/O
c W95 FAT32 (LBA) 52 CP/M 9f BSD/OS e4 SpeedStor
e W95 FAT16 (LBA) 53 OnTrack DM6 Aux a0 IBM Thinkpad hi eb BeOS fs
f W95 Ext'd (LBA) 54 OnTrackDM6 a5 FreeBSD ee EFI GPT
10 OPUS 55 EZ-Drive a6 OpenBSD ef EFI (FAT-12/16/
11 Hidden FAT12 56 Golden Bow a7 NeXTSTEP f0 Linux/PA-RISC b
12 Compaq diagnost 5c Priam Edisk a8 Darwin UFS f1 SpeedStor
14 Hidden FAT16 `/mnt/disk/home'
`/home/jblanco' -> `/mnt/disk/home/jblanco'
`/home/jblanco/.bashrc' -> `/mnt/disk/home/jblanco/.bashrc'
`/home/jblanco/.bash_profile' -> `/mnt/disk/home/jblanco/.bash_profile'
`/home/jblanco/.bash_logout' -> `/mnt/disk/home/jblanco/.bash_logout'
`/home/jblanco/.sudo_as_admin_successful' -> `/mnt/disk/home/jblanco/.sudo_as_admin_successful'
`/home/jblanco/.bash_history' -> `/mnt/disk/home/jblanco/.bash_history'
`/home/jblanco/downloads' -> `/mnt/disk/home/jblanco/downloads'
jblanco@us01:~$ cd /mnt/disk
jblanco@us01:~$ cd /mnt/disk
jblanco@us01:/mnt/disk$ ls
home lost+found

GNU nano 1.3.10 File: /etc/fstab
# /etc/fstab: static file system information.
#
#
proc /proc proc defaults 0 0
/dev/sda1 / ext3 defaults,errors=remount-ro 0 1
/dev/sda5 none swap sw 0 0
/dev/sdb1 /nas ext3 defaults 0 0
/dev/hdc /media/cdrom0 udf,iso9660 user,noauto 0 0
/dev/fd0 /media/floppy0 auto rw,user,noauto 0 0

jblanco@us01:/nas$ sudo chown -R jblanco:jblanco /nas
jblanco@us01:/nas$ sudo chmod -R 755 /nas

Repository: Change from CD-ROM to Internet

There's an easy way to fix this problem. Run the following command to open the sources.list file. You can use a different editor if you feel like.

sudo vi /etc/apt/sources.list

The first thing you'll want to do is comment out this line, by placing a # symbol before it.

deb cdrom:[Ubuntu-Server 6.10 _Ed…..

You can also use this opportunity to uncomment the universe repositories.

Now that you've updated the repository list, you will have to run this command to update the local list of available software:

sudo apt-get update

Now you can apt-get install from the internet.


Packages: See What Version of a Package Is Installed on Ubuntu

dpkg -s

See Where a Package is Installed on Ubuntu

dpkg -L
dpkg -l

Disk: List disk space usage on Ubuntu

df -Th

Ubuntu Networking: From Dynamic To Static