Thursday, June 23, 2011

Network and SysAdmin, Scripting, Debugging, O.S. Books


[System Administration]
UNIX and Linux System Administration Handbook (4th Edition) by Evi Nemeth, Garth Snyder, Trent R. Hein and Ben Whaley (Jul 24, 2010)
Essential System Administration, Third Edition by Æleen Frisch (Aug 15, 2002)
Linux in a Nutshell by Ellen Siever, Stephen Figgins, Robert Love and Arnold Robbins (Sep 29, 2009)
======================
[Network Administration]
Linux Network Administrator's Guide by Tony Bautts, Terry Dawson and Gregor N. Purdy (Feb 10, 2005)
LINUX Network Administrators Guide: 508 pages by Olaf Kirch and Terry Dawson (Jan 5, 2009)
Linux TCP/IP Network Administration by Scott Mann (Jul 26, 2001)  TCP/IP stack understand, configure and troubleshoot servers
Linux Firewalls (3rd Edition) by Robert L. Ziegler
Linux Networking Cookbook by Carla Schroder
========================
[Network and System Administration]
=====================
[Troubleshooting]
Linux Troubleshooting Bible by Christopher Negus and Thomas Weeks (Jul 30, 2004)
Debugging Embedded Linux (Digital Short Cut) by Christopher Hallinan (Aug 22, 2007)
Linux Troubleshooting for System Administrators and Power Users 2006
----
Wireshark Network Analysis: The Official Wireshark Certified Network Analyst Guide by Laura Chappell
Network Flow Analysis by Michael W. Lucas
=====================
[Performance Tuning]
Performance Tuning for Linux® Servers by Sandra K. Johnson, Gerrit Huizenga and Badari Pulavarty (Jun 6, 2005)

======================
[RHCE]
===================
[Scripting]
Perl Best Practices by Damian Conway
===================
[Linux Operating System]
----
[Parallel and Multi threading]
======================
[Enterprise]
Restful Web Services by Leonard Richardson
=====================

Saturday, June 18, 2011

Valgrind Cheat-Sheet Notes

Running Notes on Valgrind Memcheck quick-start
Original document @ http://valgrind.org/docs/manual/quick-start.html
=======================================================

1) Compile your program with following options to gcc:
            -g  //Debug mode
            -O0 //No optimization

valgrind --leak-check=yes myprog arg1 arg2

--leak-check=yes //default valgrind tool
            Turns on Memcheck profiling

2.6. Core Command-line Options

 --tool= [default: memcheck]
            Run the Valgrind tool called toolname, e.g. Memcheck, Cachegrind, etc.

-q, --quiet
            Run silently, and only print error messages. Useful if you are running regression tests or have some other automated test machinery.

-v, --verbose
            Be more verbose. NOTE: Repeating the option increases the verbosity level.
            Dump extra information on various aspects of your program, such as:
                        the shared objects loaded,
                        the suppressions used,
                        the progress of the instrumentation and execution engines and
                        warnings about unusual behaviour.

-----------------------------------------------------------------------------------------------------------------------------------------
--trace-children= [default: no]
            This is necessary for multi-process programs.
            trace into sub-processes initiated via the fork/exec system calls.

--child-silent-after-fork= [default: no]
            Use to silence confusing XML output from child when --trace-children=yes is used.

--track-fds= [default: no]
            print out a list of open file descriptors on exit.
            Along with each file descriptor is printed a stack backtrace of where the file was opened and file/socket details like name etc.

--log-file=
            send all of its messages to the specified "filename.%p.%q{PerProc-EnvVariable}"
            where %p=CurrentProcID and %q{UniquePerProcEnvVariable} is used to identify a process by the unique value of its PerProc-EnvVariable.
            something like a ProcNumber maybe.
            used along with --trace-children=yes for multiprocess applications.

[Error Related options:]
These options are used by all tools that can report errors, e.g. Memcheck, but not Cachegrind.

--xml= [default: no]
            important parts of the output (e.g. tool error messages) will be in XML format rather than plain text
            the XML output will be sent to a different output channel than the plain text output
            i.e. you also must use one of --xml-fd, --xml-file or --xml-socket to specify where the XML is to be sent. 
            Note: the destination of the plain text output is controlled by --log-fd, --log-file  and --log-socket

 --xml-file=
            Specifies that Valgrind should send its XML output to the specified file. It must be used in conjunction with --xml=yes.

--demangle= [default: yes]
            Supression files contain C++ function names in their mangled form so turn off mangling for supressions.
            Turn on supressions for human readability.

--num-callers= [default: 12]
            Use this if the stack trace is not big enough, to get more call-stack depth
            specifies the maximum number of entries shown in stack traces that identify program locations.
            errors are commoned up using only the top four function locations (the place in the current function, and that of its three immediate callers).

--error-limit= [default: yes]
            When enabled, Valgrind stops reporting errors after 10,000,000 in total, or 1,000 different ones, have been seen.

--track-origins=yes
            Helps analyse the root cause of uses of uninitialised values. Otherwise you get generic messages like "Conditional jump or move depends on uninitialised value(s)"

--error-exitcode= [default: 0]
            Specifies an alternative exit code to return if Valgrind reported any errors in the run (??as in #define TestSuiteFailed 100 in cppunit testcase??)
            ... useful for using Valgrind as part of an automated test suite, since it makes it easy to detect test cases
            for which Valgrind has reported errors, just by inspecting return codes.

 --suppressions= [default: $PREFIX/lib/valgrind/default.supp]
            Specifies an extra file from which to read descriptions of errors to suppress. You may use up to 100 extra suppression files.

 --gen-suppressions= [default: no]
            When set to yes, Valgrind will pause after every error shown and print the line:
                   ---- Print suppression ? --- [Return/N/n/Y/y/C/c] ----
            Valgrind will print out a suppression for this error. You can then cut and paste it into a suppression file if you don't want to hear about the error in the future.
            This option is particularly useful with C++ programs, as it prints out the suppressions with mangled names, as required.
            You may want to common up similar ones, by adding wildcards to function names, and by using frame-level wildcards.
            The wildcarding facilities are powerful yet flexible, and with a bit of careful editing, you may be able to
            suppress a whole family of related errors with only a few suppressions.
            ... the -v option which prints out all used suppression records.

 --db-attach= [default: no]
            When enabled, Valgrind will pause after every error shown and print the line:
              ---- Attach to debugger ? --- [Return/N/n/Y/y/C/c] ----

--db-command= [default: gdb -nw %f %p]
            Specify the debugger to use with the --db-attach command. The default debugger is GDB.

--read-var-info= [default: no]
            Memcheck gives more detailed description of illegal address in its output including variable declaration line/file and thread info.

--run-libc-freeres= [default: yes]
            Force libc.so to free its private malloc's. These are normally NOT freed but fool valgrind into logging false positives.
           Note: forcing call to libc-freeres() at the program exit MAY give problems with older buggy glibc versions.

================================================================================================================

[Setting Default Options]

Note that Valgrind also reads options from three places:
   1.   The file ~/.valgrindrc
   2.   The environment variable $VALGRIND_OPTS
   3.   The file ./.valgrindrc

These are processed in the given order, before the command-line options. Options processed later override those processed earlier; for example, options in ./.valgrindrc will take precedence over those in ~/.valgrindrc.

Please note that the ./.valgrindrc file is ignored if it is marked as world writeable or not owned by the current user. This is because the ./.valgrindrc can contain options that are potentially harmful or can be used by a local attacker to execute code under your user account.

Any tool-specific options put in $VALGRIND_OPTS or the .valgrindrc files should be prefixed with the tool name and a colon. For example, if you want Memcheck to always do leak checking, you can put the following entry in ~/.valgrindrc:

--memcheck:leak-check=yes

This will be ignored if any tool other than Memcheck is run. Without the memcheck: part, this will cause problems if you select other tools that don't understand --leak-check=yes.

================================================================================================================

[Threading Support:]
+          "your program will use the native threading library, but Valgrind serialises execution so that only one (kernel) thread is running at a time.
      ...
            threaded apps run only on one CPU, even if you have a multiprocessor or multicore machine.
     ...
            Valgrind doesn't schedule the threads itself. It merely ensures that only one thread runs at once,
            using a simple locking scheme. The actual thread scheduling remains under control of the OS kernel."
+          "if you have some kind of concurrency, critical race, locking, or similar, bugs.
            In that case you might consider using the tools Helgrind and/or DRD to track them down."

+          "On Linux, Valgrind also supports direct use of the clone system call, futex and so on. clone is supported
            where either everything is shared (a thread) or nothing is shared (fork-like); partial sharing will fail.
            Again, any use of atomic instruction sequences in shared memory between processes will not work reliably."

+          "Valgrind has a fairly complete signal implementation. It should be able to cope with any POSIX-compliant use of signals."


Signal Handling:
+          "If your program dies as a result of a fatal core-dumping signal, Valgrind will generate its own core file (vgcore.NNNNN) containing your program's state."
            "the core dumps do not include all the floating point register information. In the unlikely event that Valgrind itself crashes,
            the operating system will create a core dump in the usual way."


Memory Footprint
            "Memory consumption of your program is majorly increased whilst running under Valgrind.
            This is due to the large amount of administrative information maintained behind the scenes.
            Another cause is that Valgrind dynamically translates the original executable.
            Translated, instrumented code is 12-18 times larger than the original so you can easily end up with 50+ MB of translations when running (eg) a web browser."
==================================================================================================================
Valgrind:
  • memcheck, 
  • cachegrind, 
  • multi-threaded

Valgrind Article on Memory Debugging and Profiling Howto
Valgrind Howto