SoftSource Solutions Pte Ltd

Getting your Business Edge with Big Data

webmaster — Wed, 13 Jun 2012 10:06:59 +0000

Join us in our IBM System x and IBM Infosphere BigInsights seminar

Click on the image to register now!

Big Data is more than simply a matter of size; it is an opportunity to find insights in new and emerging types of data and content, to make your business more agile, and to answer questions that were previously considered beyond your reach.

IBM Infosphere BigInsights bring the software solution to Big Data, while IBM System X brings the hardware and system software solution to Big Data.

IBM Systems X capabilities in the areas of fast setup, systems networking, more efficient cooling, faster access to data using flash storage (eXFlash), and a wide range of processor performance choices enables customers to build and grow the right Big Data systems according to need and demand.

Speaker Information

Tan Yik-Chuan

Tan Yik-Chuan is a member of the IBM ASEAN Client Technical Professional team for Information Management and PureApplication System. As an IBM Certified ITS Specialist, he specializes in enterprise data management, data governance and Big Data technology. His 16-year career spans development and technical sales of products for databases, supply chain management software and geographic information systems. Yik-Chuan holds a master’s degrees in Computer Science from University of Maryland (USA) and a bachelor’s degree from National University of Singapore.

Cheng Jang Thye

Cheng Jang Thye is currently a Systems Architect with the IBM Systems and Technology Group (STG). Current responsibilities include providing pre-sale engagement support for Smarter Business Analytics and infrastructure consultancy to key customers in Singapore. He has more than 20 years of professional experience in a wide range of technology areas including data center networking, cloud computing, security, J2EE, enterprise UNIX systems, enterprise application integration, electronic commerce and distributed computing.

Scripting Tips #2: How to compare 2 sets of files to see if they are the same?

webmaster — Fri, 03 Feb 2012 03:51:55 +0000

Excerpts from Simplify, Automate, Liberate! For Unix/Linux System Administrators

written by Gerald Yong

Problem

You have 2 different folders, and you want to compare the files in the folders to see if they are the same or different. The files have the same names in both folders.

Solution

Compute a fast hash or checksum for all the files would be a faster method.

#!/bin/ksh

DIR1=/export/home/geraldy/scripts/03/dir1
DIR2=/export/home/geraldy/scripts/03/dir2
CKSUM_FILE=cksum.txt

gen_checksum() {
typeset TARGETDIR
TARGETDIR=$1

cd ${TARGETDIR}
cksum * > ${CKSUM_FILE}
}

gen_checksum ${DIR1}
gen_checksum ${DIR2}

diff ${DIR1}/${CKSUM_FILE} ${DIR2}/${CKSUM_FILE}

Discussion

You may be tempted to use diff to compare each file with its twin in the other folder to see if there are differences. However this will take much longer.

cksum will generate a unique number for specified files. Because cksum can generate the number quickly, it can be used as a quick way to see if 2 files have the same checksum, and hence determine if they are the same.

The cksum method is a fast method of comparing files, but do note that 2 files with the same contents but different last modified dates will still have the same checksum.

# cksum *
735629196 151 1.ksh
3476470671 145 IP.txt
515424607 201 a.out
3407276374 223 ansichart.ksh
2496520640 112 arithmetic.ksh
217785708 399 arrowkeys.ksh
3210263215 139 ask_userinfo.ksh
3852637783 160 awk1.awk
3245121712 201 check_digit.ksh
3171192017 389 child.ksh
1248279931 229 chk_file_exists.ksh
552807997 116 chk_ora_listener.ksh
3997812998 206 chk_root.ksh
4060654239 291 chk_user.ksh
1701067367 390 chk_user2.ksh
#

The script above uses cksum to generate checksums for all the files in each directory and stores them in a text file in each directory. It then uses diff to compare the text files, so that you can tell which files have different checksums.

Need more information? Looking for expert advice to implement a mid-range to
enterprise level backup/storage solution in your organisation?

Click on the image below to contact us!

Scripting Tips #1: How to check if a command has run successfully

webmaster — Thu, 02 Feb 2012 10:18:45 +0000

Excerpts from Simplify, Automate, Liberate! For Unix/Linux System Administrators

written by Gerald Yong

Problem

You have a script, and inside the script you call an external command. You want to know if the command has succeeded or failed, and take appropriate action. For example, you may want to know if the scp command that copies a local file to a remote destination was successful in the following example:

$ cat utl_chk_scp.ksh
#!/bin/ksh
#
# Copies a local file to a remote location.

LOCAL_FILE=/opt/app/log.txt
DEST_CONN=john@server1
DEST_PATH=/opt/app/logs
SCP=/usr/bin/scp

echo “Performing scp of ${LOCAL_FILE} to ${DEST_CONN}:${DEST_PATH}”
${SCP} -p -q ${LOCAL_FILE} ${DEST_CONN}:${DEST_PATH}

Solution

Obtain the exit code of the command from the shell variable $? .

${SCP} -p -q ${LOCAL_FILE} ${DEST_CONN}:${DEST_PATH}

# Check if the exit code from the previous scp command is 0
# (for success) or not 0 (for failure).
if [ $? -ne 0 ]; then
echo “CRITICAL: Failed to scp ${LOCAL_FILE} to dest ${DEST_CONN}.”
exit 1
fi

Discussion

If the external command returns a message upon success or failure, then you can store the output of the external command into a variable using a backtick operator.

OUTPUT_MSG=`command`

Otherwise, you can capture the exit code of the command using the shell variable $? .

This only works if the command adopts standard behaviour of returning an exit code of 0 when the command is successful, and non-zero when there are any errors during the execution of the command.

For the above example:

$ cat script.ksh
#!/bin/ksh
#
# Copies a local file to a remote location.

LOCAL_FILE=/opt/app/log.txt
DEST_CONN=john@server1
DEST_PATH=/opt/app/logs
SCP=/usr/bin/scp

echo “Performing scp of ${LOCAL_FILE} to ${DEST_CONN}:${DEST_PATH}”
${SCP} -p -q ${LOCAL_FILE} ${DEST_CONN}:${DEST_PATH}

if [ $? -ne 0 ]; then
echo “CRITICAL: Failed to scp ${LOCAL_FILE} to dest ${DEST_CONN}.”
exit 1
fi
exit 0

It’s always a good practice to have your scripts return 1 or some non-zero value when there is an error, and 0 when the script has run successfully. That way, if this script is called from within another script, that script can also check the exit code to see if this script has executed successfully.

Note that if you run multiple commands, then $? will store the exit code of the last command run. So if you wish to check for the success or failure of each command, you will need to test $? after each command has been executed.

# Run 1st command.
${SCP} -p -q ${LOCAL_FILE1} ${DEST_CONN1}:${DEST_PATH1}

if [ $? -ne 0 ]; then
echo “CRITICAL: Failed to scp ${LOCAL_FILE1} to dest ${DEST_CONN1}.”
exit 1
fi

# Run 2nd command.
${SCP} -p -q ${LOCAL_FILE2} ${DEST_CONN2}:${DEST_PATH2}

if [ $? -ne 0 ]; then
echo “CRITICAL: Failed to scp ${LOCAL_FILE2} to dest ${DEST_CONN2}.”
exit 1
fi

This method also works if you have many functions in your script. In the following example, there are 3 functions, each will return 0 if the function runs successfully, and 1 if the function has an error. The main program calls each function in turn, and checks their exit code. If any of the function fails, the program exits with an exit code of 1.

check_status() {
# If success
…
return 0
# else
…
return 1
# fi
}

update_file() {
# If success
…
return 0
# else
…
return 1
# fi
}

send_email() {
# If success
…
return 0
# else
…
return 1
# fi
}

# Main program.
check_status
if [ $? -ne 0 ]; then
exit 1;
fi

update_file
if [ $? -ne 0 ]; then
exit 1;
fi

send_email
if [ $? -ne 0 ]; then
exit 1;
fi

Need more information? Looking for expert advice to implement a mid-range to
enterprise level backup/storage solution in your organisation?

Click on the image below to contact us!

Finding Simplicity in Data Complexity

webmaster — Thu, 01 Sep 2011 06:55:12 +0000

Come join us for a workshop on IBM Storwize V7000 & Symantec Filestore!

New Corporate Website

webmaster — Mon, 09 Aug 2010 04:19:10 +0000

Softsource Solutions is please to announce the release of our corporate website. The 4 pages website, which have served us well for many years has been replaced to bring to you a bigger and better web experience.

Workshop: Securing, Protecting & Archiving (SPA) of your Enterprise data

webmaster — Fri, 29 Jan 2010 00:40:57 +0000

On 29^th of January SoftSource together with Symantec organized a workshop with focus on the following areas: Securing, Protecting and Archiving your organization data. The event was held at Singapore Recreation Club (SRC) from 2-5pm.

At the exclusive workshop we shared with our customers on how Symantec products, together with Softsource expertise can help protect your data and resolve your issues related to:

Data Lost Protection (DLP) – Data Loss Protection suite that focus on creating enterprise policies that detect, monitor and protect your data in your organization endpoints and networks.
Enterprise Vault (EV) – Comparison of enterprise vault with exchange 2010 and how you can make use of the new features in EV 8.0 to enhance the protection of your archival data
NetBackup (NBU) and Backup Exec (BE) – New features of the backup solution to guarantee availability of your backup data and promise granular and easier restore of your mailboxes and email message.

Here’s some highlight of the day.