Friday, February 6, 2015

C Pointers in a Simple English

Everyday Analogy

What is a pointer?

A pointer is a 'piece of paper' which stores the 'phone no' of a 'target' object .
Using the pointer we can access the target.
Since we can easily change the target, we can create programs which don't need to know what variable/object they're operating on. They just operate on the variable indirectly through the pointer and don't care what the target is

Variables are like trees, they're rooted to one place
Pointers are like cheetahs, they're agile and dangerous.

TODO : Replace the kludgy diagram below with a scanned proper hand-drawn version.

Imagine a table with rows and columns, let's say this represents a "logical RAM".
Each cell can store data. So each cell can act as a variable.
Some of these variables could store locations of other cells instead of just number values.

So the cells are numbered left-to-right and top-to-bottom from 1-12 in the figure below:


So the 3rd cell stores the value 150 in it. And the 11th cell stores the value 200 in it.
So the 7th cell stores the address '3' to point to the 3rd memory cell (containing the value 150 in it).
And the the 10th cell stores the address '11' to point to the 11th memory cell (containing the value 200 in it).

Everyday Analogies:

A) Telephone Number Analogy :

We regularly use pointers in everyday life when we use telephone numbers.
 [Note :
  a)Telephone no.s give 2-way access whereas a pointer gives only 1-way access.
  b)Usually the no. of links is 1 to 3 in an access path.
 ]

Find any Person using your Friend Network (a la 6 degrees of separation):

Let us say Tom wants to tell his friend something important.
He can either physically go and talk to him or he can phone him.

Lets assume Tom phones Jerry.
   Tom ---> Jerry

Tom doesn't have Jerry's no. Tom calls up 1 to get Jerry's no.
   Tom ---> 1 ---> Jerry

Tom follows the "Six Degrees of Separation Rule" as he knows that someone in a path going outward will know Jerry's no.
   Tom ---> 1 ---> 2 ---> .... ---> N ---> Jerry

Here Tom calls Jerry by
 a) Firstly getting the contact no. directly or indirectly from 'a friend-of-a-friend-of-a-friend'.
 b) Secondly Dialling the number.

B) Links of a Chain : 

Here the intermediate links of the chain can change the destination similar to an operator diverting your call to
the concerned person in an organisation.

Say Tom wants some info (from Jerry), but N on the path knows that Mickey is a better source of info and gives Mickey's number instead.

 a)Original path:
   Tom ---> 1 ---> 2 --->...N ---> Jerry

 b)Redirection:
   Tom ---> 1 ---> 2 --->...N ---> Mortimer

We can remove a single link from anywhere in the chain and re-create the (slightly smaller) chain by reattaching the links near the break in the chain. There is a minimal cost of inserting new links or removing existing links in the chain. This is exactly the property which allows efficient linked lists and other data-structures to be used without being stuck with unwieldy arrays.
It takes a lot of copying to cover up the "hole" in an array if we need to delete or insert at the front or middle.
But pointers just require reassigning the links fix the "break" in the chain.

C) Train Shunting :

This is similar to shifting tracks for a train, we can easily divert the train to whatever track by using the shunting mechanism.

D) Postal Address :

Though not as precise and neat as the telephone number analogy you can think of a pointer variable as an envelope with the address of a person on a paper inside.

When do we use Pointers?

 a)When we want to randomly point to and access different areas of memory (say a sequence of objects).
 b) Access a specific memory location. (accessing a port or a vdu buffer)
 c) Create an anonymous object.
 d) Save our position in a sequence.
 e) Avoid needless copying/moving of data items in a data-structure by just adjusting few pointers.

A pointer can point somewhere (an object) or anywhere (an address).
 It can be repositioned to point elsewhere.

 Access the object contents by location (Like voodoo)
 Re-target
 Insulate knowledge of identity of target object. (Bearer Cheque, Lottery winner)
 Anonymous objects (using new or malloc )
 Iterating over a Sequence

  ptrvar = &Object1;    //targeting the pointer
  ptrvar           Object1
 [ 1000 ] -----> [........] location 1000

  ptrvar =  &Object2;    //retargeting the pointer
 [ 2000 ] -----> [........] location 2000


We need pointers to keep track of our position as we access different locations.
Everyday usage :
The index finger 'points' to our current location in a phone-directory as we rapidly scan through the entries.
Website URLs, Postbox numbers or Postal addresses.

 We want to call up all people with the prefix 2375XXXX we just start at 23750000 and then
 keep adding 1 to the no. till we reach 23759999
 We could call up all people with 2375XXXY where Y is an even no. and so on.
 We can randomly dial any no. without knowing the person at the other end (anonymous object)

How do we use Pointers?

a) Example code to understand Pointers : C Source Code/Pointers 

b) C Code to traverse and insert a node in a linked-list:
 typedef struct list {
   int data;
   struct list *next;
 }List;

 void printlist(List *first)
 {
  List *tmp;
  for(tmp = first; tmp != NULL ; tmp = tmp->next ) { // step through the sequence
      printf("%d", tmp->data);
  }
  return ; 
 }

 int main()
 {
  int x=1,y=2,z=0;
  int num[3] = { 1, 2, 0 };
  List *head = makelist();    //returns an anonymous list of objects ->[10]->[20]->[30]->null
 
  add(&x,&y,&z);            // z = 3
  add(&num[0],&num[1],&num[2]);        // num[ 1, 2, 3]
  printlist(head);

  return 0;
 }

 void add(int *a,int *b,int *c)
 {
  return *c = *a + *b;
 }
===================================================================

Location access using Pointer or Array Notation:


int iarr = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 }; //Note : arrays are 0-indexed in C
int * iptr = &iarr[0];   //store address of 1st location of array in iptr
*iptr = 42;  //get the address from envelope using star operator, go to location and assign '42' number

iptr[0] = iptr[9];  // read last integer of array and assign to 1st cell of array (using array notation)
*(iptr+0) = *(iptr+9);  //Same as previous code we're just using pointer notation (instead of array notation).


See Also : 
  1. Ted Jensen C Pointer Tutorial : http://pweb.netcom.com/~tjensen/ptr/pointers.htm
  2. The 5 Minute Guide to C Pointers : http://denniskubes.com/2012/08/16/the-5-minute-guide-to-c-pointers
  3. "Understanding Pointers in C" by Yeshawant Kanetkar (BPB Publications).
  4. "Understanding C by Learning Assembly" : https://www.hackerschool.com/blog/7-understanding-c-by-learning-assemblyhttps://www.hackerschool.com/blog/7-understanding-c-by-learning-assembly
  5. Learning C with GDB : https://www.hackerschool.com/blog/5-learning-c-with-gdb
  6. Super Tutorial on Linux Memory Management : http://www.linux-tutorial.info/modules.php?name=MContent&pageid=261
  7. Internals of C++ and Unix : http://techtalkies.blogspot.com/2007/07/good-articles-on-internals-of-c-and.html

Tuesday, August 26, 2014

Installing and Testing out your own crowd-sourced App with PyBossa


Testing PyBossa with a Virtual Machine : http://docs.pybossa.com/en/latest/vagrant_pybossa.html

Install Virtualbox and Vagrant first.

 
Problem : The 'vagrant up' fails on WinXP while installing 'hashicorp/precise32' dependency.
Following error message is seen : 

Cause :  This is caused by missing or failure in running bsdtar.exe.
For More Details see : https://github.com/mitchellh/vagrant/issues/3869.
Solution:
1) Install Mingw with bsdtar.exe
2) "However, installing MinGW for the bsdtar.exe, brought up another issue.
bsdtar.exe wouldn't run because it needed 'msys-lzma-1.dll' but 'msys-lzma-5.dll' was available.
Just copied and renamed to expected filename and it worked!!"


Now precise32 installs and 'vagrant up' succeeds.

Next step

 
Problem : Running the 'source vagrant_start.sh' script fails
Cause : bash requires usage of ./scriptname to execute script in current directory.
Solution : Just changing the docs line 'source vagrant_start.sh' to 'source ./vagrant_start.sh' fixes the problem.

Problem : 'source ./vagrant_start.sh' still fails
Cause : It gives errors for trailing ^M characters in the script, DOS line-feed chars.
Solution : Editing the file and removing trailing ^M characters solves the problem.

Use your web browser to browse Pybossa welcome page with this URL 'http://localhost:5000'

Thursday, July 17, 2014

test discourse

1) Configure Discourse Server Settings as described here: http://eviltrout.com/2014/01/22/embedding-discourse.html

2) The Discourse comments should appear here ===>

Tuesday, July 1, 2014

Test Google Transliterate Widget


Type in Indian languages (Press Ctrl+g to toggle between English and Hindi)
Type in

Body

Tuesday, June 17, 2014

Test Amara Widget


Amara Test Video for Easy Access to Amara Subtitling and Interactive Transcript on your blog or website



Usage:

1) Click on "amara" button on video toolbar to view the different language subtitles for the video
2) Click on "Transcript" button to view the interactive transcript for whole video.
    Also shows "Search Toolbar" box to highlight any word/phrase in the text.
    Skip to any point in the video timeline by simply clicking on any word in the interactive transcript.
    Can be used to extract quotable quotes very easily for sharing on social media (with time position).
3) Click on "CC" button to toggle subtitles.
4) Click on "English" button to get menu to
 a) "Improve these subtitles" i.e. add new or update existing subtitles (need to login).
 b) "Embed Code" to embed the amara widget in any other page.
 c) "Download Subtitles"
 d) Switch languages.

Rajiv Malhotra Test Video

Friday, February 14, 2014

Super way to create running notes on any ebook using Kindle

Super way to create running notes on any ebook using Kindle

Advantages:
a) Use any spare time anywhere to read and create my own notes on any kindle ebook.
b) Well worth the price of a kindle ebook.
c) Many times it's difficult to remember detailed stuff that you read quite some time back.
d) With this method you end up spending at most 2 times effort but you can keep building on it incrementally.
e) However, since it's mostly automated it reduces typing effort.
f) The draft and final notes are best kept in a wiki.
g) Periodically update onto blog.

Note: I've written to the author of bookcision bookmarklet to add copy of user notes in addition to highlighted text. He just replied back that the feature is being added in next version of the tool. Till then it's possible to extract "Notes" by copy-pasting from kindle web-page.

Buying eBooks on Amazon.com:
Note: As of 13-Feb-2013
a) You will be Re-directed from India amazon..in to US site amazon.com for buying kindle ebooks.
b) You need to use international credit-card to buy the same.
c) You can pay in Rupees for the same.
d) You may need to change the country of residence.
e) Of course you can always get "sample ebooks" or "free ebooks" on your Smartphone/PC/CloudReader.
f) Just click Deliver to each of the specific devices.

Howto Annotate your Kindle ebook for running Notes:
a) Install Kindle-for-PC, Kindle-Android-App, Kindle-Cloud-Reader (best for quickly editing notes).
b) Add bookcision bookmarklet to firefox/chrome browser : http://www.norbauer.com/bookcision
c) Highlight the Chapter/Topic in addition to text which you want to annotate.
    You can also add a note-to-self to the highlighted text.
d) Highlights get stored on your kindle account : https://kindle.amazon.com/your_highlights
e) Click bookcision bookmarklet to copy highlights to clipboard. This will copy only highlighted text
f) To copy notes part of your highlights page "View Source(Ctrl-U)" and manually copy contents 'noteContent' tag to your wiki.
g) Copy the notes to your Tiddlywiki and edit for formatting.

Sample:
http://learning-notes.tiddlyspot.com/#[[Mastering%20Redmine%20Book%20Notes]]

Install:
a) Kindle Cloud Reader : https://read.amazon.com
b) Kindle App for Android :  https://play.google.com/store/apps/details?id=com.amazon.kindle
c) Kindle For PC : http://download.cnet.com/Kindle-for-PC/3000-20412_4-75185974.html

See Also:
a) Howto Install Kindle for PC : http://www.howtogeek.com/howto/9192/read-kindle-books-on-your-computer-with-kindle-for-pc


Tuesday, December 24, 2013

LXR Source Code Referencing using iframe

Reference by File,Line No:

Reference by Keyword:


LXR - Source Code Cross Referencing Tool: http://lxr.linux.no
WordPress Hosting solutions: http://wordpress.org/hosting

STL Video Tutorial by the Microsoft STL maintainer Stephen Lavavej

Going Deep into STL with Stephen Lavavej (Video)
Inside STL with Stepan Lavavej (Video)
C++0x Features in VC10 

Todo: Viewing and analysis to follow
Note: Thanks to my team-mates (SriVasala and Ravi H.) for giving the link to the video

Resources for Exploring the Python Standard Library

Thursday, July 28, 2011

Good Articles on Internals of C++ and Unix

A) C as a glorified assembler:

"C a glorified assembler" or "C a generalized assembler"

Actually I only really started understanding C when I understood 2 things about it.
1) Anything you code in C finally translates to assembly language.
2) How your compiler interacts with the O.S. loading/linking mechanism.

You can easily test this.
Use the 'gcc --save-temps ' feature of your gcc compiler to see the assembly language output.

C program to assembly code translation
>>>>>>>>
C program
>>>>>>>>
extern int printf(const char *, ...)
int main()
{
 printf( "Hello, World!\n");
 return 0;
}

#Retain temporary intermediate files generated during preproc, compiling, assembling etc.
gcc --save-temps test.c

>>>>>>>>
Pseudo-Assembly (see link for more accurate example)
>>>>>>>>
.data                              # data section starts at say 0x100
msg ds "Hello, World!\n"           # define string with label as msg and content as "Hello, World!\n"

.text
push sp                            # save the current stack pointer
push msg                           # push the function parameter onto stack
call _printf                       # make the library call
ret   0                            # essentially return 0 as errorvalue to bash/cmd.exe

>>>>>>>>

Since the C language is portable from one platform to another it kind-of acts like a virtual machine (without byte-code generation of-course).
The same C program will compile on different machine-compiler pairs to the specific assembly language and O.S. of the platform in question.

This makes for  chip/hardware specific assembly(Intel x86 or Motorola or ARM ...) portability.

Write Great Code :

What happens when you run your program?
The dynamics from Compilers to O.S. to Assembly language to Memory to CPU.
This book covers it all

You have 2 choices:
1) You can choose to spend years garnering small nuggets of internals of your Program and how it runs on your machine. You'd need a few books each on Operating System, Linker Loader, Assembly Language, Compiler Design, Hardware.
Next would be loads of patience to connect all this into a chimera/frankestein-of-sorts (horribly put together)
OR
2) You could read it all here - Integrated into a beautiful and coherent whole written by a master.

Write Great Code, Volume 1: Understanding the Machine
Write Great Code, Volume 2: Thinking Low-Level, Writing High-Level
The Art of Assembly Language


Refer:
  1. C to Assembly Translation Article from EventHelix
  2. Inside the C++ Object Model : http://techtalkies.blogspot.in/2007/07/stub-bookreview-c-object-model-by.html
  3. X86 Calling (and stack cleanup) conventions: http://en.wikipedia.org/wiki/X86_calling_conventions
    X86 Disassembly Wikibook (download as pdf): http://en.wikibooks.org/wiki/X86_Disassembly
==================================================================

B) Interaction with O.S. loader/linker/virtual-memory:

The specific C compiler would also link the O.S. specific C runtime properly.

So gcc which is a portable compiler would link in.
a) mingw C runtime on Win32
b) linux C runtime on Linux.

The same C program would also convert to correct assembly/runtime on
Watcom C++, Visual C++ compiler, AIX, HP/UX compilers etc.

Note:
a) printf() comes from the C Standard Library
b) C runtime library contains the startup code which executes before and after main

Refer:
1) C++ runtime environments on HP-UX
2) Internals of C/C++ compiler implementations
3) How to use debug C runtime to debug your application
4) How to write a minimal Kernel in C (a very simple Hello world program which also describes the libc C runtime library)
5) Understanding System Calls
Note:
The C standard library calls like open(), close(), putc(), getc() etc actually just forward the call to O.S. system calls in the linux kernel. The C runtime actually maps a stdlibrarycall to an interrupt vector i.e. system call index number in a lookup table. Then it uses a software interrupt to call the system call (transfers from user mode to kernel mode and back)

==================================================================

These are a set of articles that take a look at what happens inside C, C++, Unix internals as we compile and run our programs. The info here is invaluable to debug compile errors, runtime errors and memory issues. You can go through these in your spare time. Take a print out to read in your spare time and go through these articles.

  1. What Happens When You Compile and Link a Program
  2. What a Compiler Turns Your C Code Into
  3. Virtual Base Classes Implementation
  4. How the C++ compiler mangles/decorates function namesUnix And C-C++ Runtime Memory Management For Programmers
  5. Under The Hood Look At Operating Systems Internals with Windows and Linux : http://techtalkies.blogspot.in/2010/08/operating-systems-and-linux.html
  6. mmap is not the territory (Part 1): http://techtalkies.blogspot.com/2010/09/mmap-is-not-territory-or-mapfail-sigbus.html
  7. mmap is not the territory (Part 2): http://techtalkies.blogspot.com/2010/09/mmap-is-not-territory-part-2.html

==================================================================

Wednesday, July 13, 2011

Remotely editing files with ssh, sshmenu, bcvi and vim

No need to install your favourite vim settings etc on the remote server just use bcvi to start gvim on your workstation.
It uses scp internally to create an editing session from workstation to server

Transparent multi-hop ssh
sshmenu setup howto
Remotely editing files with bcvi and vim
Editing remote files in vim via scp

Browsing XSLT with Vim by adding a Custom Language to Ctags

XSLT Source Code Browsing with Vim and Ctags

Creating a tags file

1) Test out the regular-expressions for your custom language:
$> egrep 'pattern' *.xsl
2) Copy-paste custom language with the above regular-expressions into ~/.ctags.
$> vi ~/.ctags
--langdef=EXSLT
--langmap=EXSLT:.xsl
--regex-EXSLT=/--regex-EXSLT=/--regex-EXSLT=/$> cd srcdir
$> ctags -R *
$> vi ./tags        #check for tag-entries containing the search patterns
$> vi test.xsl     #Use Ctrl-] to jump from a pattern usage to its definition, Ctrl-T to jump back
Note:
1) Above solution for templates only uses xpath of match="xyz".
The search-key for the template (displays taglist for templates with same xpath).
Potentially a template is uniquely identifiable using match AND mode:

Monday, July 11, 2011

Apprenticeship Patterns: Patterns and Anti-Patterns of Problem-Solving

The Intellectual Side of Problem-Solving:

(Borrowing heavily from AI terminology)
Imagine a 3D Cube containing the start-state, the goal-state, the problem-space and the solution-paths.
You're essentially scanning a problem space for a solution i.e. a combination of sub-steps which finally lead to goal.
1) Explore the problem space first i.e. try out things
2) Identify what worked or is promising and extend the branches into the sub-problem spaces
3) Recurse over the sub-branches until you've covered enough of the problem-space.
4) If the volume of the problem-space is vast it may help to BackTrack from Goal towards the problem state.

This reduces the volume of problem-space to be probed for possible solution-paths. This is known an "Alph-Beta" Pruning of the (Decision??) Tree.
//TODO: Add Diagram of 3D Problem Space and Decision(??) Tree.


In Nature, Lightning follows a very similar method of Back-Tracking.
It seeks out the easiest path between cloud and ground through an insulating volume of air.
Actually there is an initial weak backtracking path from ground to cloud, followed later by a strong return path.
//TODO: Add picture of Lightning fingers

The Emotional Side of Problem-Solving:

Basicially we must first ask "Why" is problem-solving so important?
Not all problems we face are life-threatening to require immediate attention.
So then what is it that makes solving them so important in the first place.

Refer:
1) Psychology of Computer Programming by Gerald Weinberg
2) Emotional Intelligence

Experience - coming to terms with "immediate failures" vs. "future pay-offs"

1) Try out many things to get a better solution or a solution which appeals to your way of doing things.
2) All the things you try won't/can't succeed at least immediately.
   Mostly many of these "dead-end branches" are just waiting for the right time to flower. You're actually building up a fund of ideas and efforts which will help you overcome some other obstacle.
   Example:
   Unix was a result of dissatisfaction with implementing Multics which itself was a replacement for some other O.S.
   Ken Thompson went on to implement ideas for distributed O.S in CODA, Plan9 etc. 30-40 years later these lessons learned and 'the-roads-tried-but-not-taken' have resulted in writing a 'go' language.
   It's used in Google which puts those same ideas refined over time into concrete working programs.

3) Watching my son getting frustrated when his cycle got stuck in a corner.
   He'd start screaming and banging the cycle into the wall.
   Beating it with red-face and small fists.
   The tears of frustration and incomprehension as to why the universe conspired against his cycling.

4) This felt so much like my own frustration with my own efforts.
   The perceived low return-on-investment of the "meagre" results.
   So many ideas half-implemented, unimplemented and ALL the wonderful things not even attempted i.e. 'the-road-not-taken'.
   The universe almost conspiring to thwart each hope and attempt at magically succeeding at all things.

5) Frustrations comes from other people's expectations.
   And more so our own awareness of not "measuring-up" to their and our own expectations.
   Perfectionism adds heavily to an already sticky situation.
   A feeling that time is running out.
   The impending joining of ranks with washed-out oldies.
   A feeling of helplessness and hopelessness that things can't/won't change no matter what.  
   Unable to leave coding and unwilling to change to management (manipulating people not machines).
   Fear of becoming obsolete, of not living up-to the dreams, hopes and self-expectation built up over the years.

----

Patterns: The Middle Path between the Emotional and Intellectual Sides of Problem-Solving

"Why patterns work?!!" AKA "All I needed to know about Patterns I learned in Kindergarten"

Patterns force you to Think and NOT concentrate so much on the Feeling part (which is the cause of frustration)
a) Read, understand, see how the Pattern fits the Problem.
b) Re-evaluate the problem itself.
c) Try carrying out/coding the Pattern.
d) Customizing the Pattern to the situation.
e) Utilize your Feeling side by reflecting on aesthetics as pattern application is a very qualitative activity.

Patterns are primarily a set of ready-made formulas (cookbook-style) for successful problem-solving.

Raise Flagging Hopes:

These are also sufficiently magic-endowed by hearing good things from others etc. This helps you get over the inertia of hopelessness or feeling stuck at a problem. "Who knows this just might work - I've nothing to lose."

Abstract the problem solving process.

De-personalise the problem takes the bite out of it.
If the trial fails it's just the Pattern failing NOT you.
You're fore-warned to try out different patterns for different circumstances.
So you just try some other Pattern or just do something else.
Since the Pattern comes from outside, you give yourself time to understand it.
This is the crucial part where you become accepting of time and effort to customize the Pattern to your particular situation.

Conclusion:
Patterns are very effective Devices for Distraction from Result and Focus onto Process instead.

  1. Failure Modes of Hacker Schoolers : https://www.hackerschool.com/blog/66-four-failure-modes-of-hacker-schoolers

Friday, July 1, 2011

How-to Repair an Oracle Database using Putty, SQL Developer GUI and some trial-and-error

I had a problem earlier when I made some changes to the test/dev DB and didn't know how to revert back to correct state.

Issue:
I couldn't delete a row in a table containing incorrect data as the row's primary key was acting as a foreign key in some other table. If I deleted this row it would violate the integrity of the other table(s).

Process:
So I took a backup of the entire DB (just in case I needed it  and I did need it!!).
Dropped the constraints, deleted all the related rows (or something like that)
Recreated the constraints and the DB was restored to correct state.
In the process I learned quite a bit about DB's.

Tried out individual commands in the GUI.
Then copied multiple commands into a script.
Ran the scripts, corrected any errors till I got a working script.
Liberally used transaction and rollback to help in the trial-and-error process.

----
I don't remember the specific details but basically
I used a GUI DB browser (SQL Developer) to connect to a Remote DB by port forwarding from my Unix machine.

Working with command-line client is ok if you're really good at it and know all the command syntax etc.
But using a GUI browser really speeds things up as it can manage things like transactions, rollback etc all at the click of a button.

Database GUI SQL Developer allows you to
+ Try out individual commands (with rollback) to see if you're getting the right results.
+ Copy paste small commands into a bigger sql script (with start transaction at start and commit or rollback at script end)
+ Browse a remote DB in GUI mode using a DB connection (using ssh tunnel),
+ backup the entire database (for safety)
+ maybe try out things on a local database first, before attempting the same on the actual DB.

You can export the original DB. Import into a local DB. Try out the operations to check it works.
It may also help to create try out your modifications on a duplicate DB on remote machine instead of touching the problematic DB directly.

1) SQL Developer Setup and Tunneling (port forwarding) instructions are available here.
http://www.cs.tau.ac.il/~boim/courses/databases2009/slides/moreinfo/connection-guide.htm#_Toc150420090
http://www.madirish.net/?article=152
2) You may need to tweak a few things on your Unix machine to get the tunnel working though.
3) Finding somebody in your/other teams who's good at Databases will be a big help.
    Local person who can see what the problem is and give quick suggestion is always better.
4) Searching on error messages would help find howto's for fixing the problem.
5) Search error messages in forums like stackoverflow.com will help to quickly find the info or relevant links.

HTH.
----
Oracle SQL Developer

PDF - Oracle SQL Developer User Guide
Oracle SQL Developer at wikipedia
Oracle SQL Developer Tutorial
Connecting Oracle SQL Developer to MS SQL Server
How-to Export Tables/Entire DB with Oracle SQL Developer

Oracle SQL*Plus command line interface (CLI)

SQL*Plus CLI FAQ
SQL*Plus CLI command reference
SQL*Plus Tutorial

Thursday, June 23, 2011

Network and SysAdmin, Scripting, Debugging, O.S. Books


[System Administration]
UNIX and Linux System Administration Handbook (4th Edition) by Evi Nemeth, Garth Snyder, Trent R. Hein and Ben Whaley (Jul 24, 2010)
Essential System Administration, Third Edition by Æleen Frisch (Aug 15, 2002)
Linux in a Nutshell by Ellen Siever, Stephen Figgins, Robert Love and Arnold Robbins (Sep 29, 2009)
======================
[Network Administration]
Linux Network Administrator's Guide by Tony Bautts, Terry Dawson and Gregor N. Purdy (Feb 10, 2005)
LINUX Network Administrators Guide: 508 pages by Olaf Kirch and Terry Dawson (Jan 5, 2009)
Linux TCP/IP Network Administration by Scott Mann (Jul 26, 2001)  TCP/IP stack understand, configure and troubleshoot servers
Linux Firewalls (3rd Edition) by Robert L. Ziegler
Linux Networking Cookbook by Carla Schroder
========================
[Network and System Administration]
=====================
[Troubleshooting]
Linux Troubleshooting Bible by Christopher Negus and Thomas Weeks (Jul 30, 2004)
Debugging Embedded Linux (Digital Short Cut) by Christopher Hallinan (Aug 22, 2007)
Linux Troubleshooting for System Administrators and Power Users 2006
----
Wireshark Network Analysis: The Official Wireshark Certified Network Analyst Guide by Laura Chappell
Network Flow Analysis by Michael W. Lucas
=====================
[Performance Tuning]
Performance Tuning for Linux® Servers by Sandra K. Johnson, Gerrit Huizenga and Badari Pulavarty (Jun 6, 2005)

======================
[RHCE]
===================
[Scripting]
Perl Best Practices by Damian Conway
===================
[Linux Operating System]
----
[Parallel and Multi threading]
======================
[Enterprise]
Restful Web Services by Leonard Richardson
=====================

Saturday, June 18, 2011

Valgrind Cheat-Sheet Notes

Running Notes on Valgrind Memcheck quick-start
Original document @ http://valgrind.org/docs/manual/quick-start.html
=======================================================

1) Compile your program with following options to gcc:
            -g  //Debug mode
            -O0 //No optimization

valgrind --leak-check=yes myprog arg1 arg2

--leak-check=yes //default valgrind tool
            Turns on Memcheck profiling

2.6. Core Command-line Options

 --tool= [default: memcheck]
            Run the Valgrind tool called toolname, e.g. Memcheck, Cachegrind, etc.

-q, --quiet
            Run silently, and only print error messages. Useful if you are running regression tests or have some other automated test machinery.

-v, --verbose
            Be more verbose. NOTE: Repeating the option increases the verbosity level.
            Dump extra information on various aspects of your program, such as:
                        the shared objects loaded,
                        the suppressions used,
                        the progress of the instrumentation and execution engines and
                        warnings about unusual behaviour.

-----------------------------------------------------------------------------------------------------------------------------------------
--trace-children= [default: no]
            This is necessary for multi-process programs.
            trace into sub-processes initiated via the fork/exec system calls.

--child-silent-after-fork= [default: no]
            Use to silence confusing XML output from child when --trace-children=yes is used.

--track-fds= [default: no]
            print out a list of open file descriptors on exit.
            Along with each file descriptor is printed a stack backtrace of where the file was opened and file/socket details like name etc.

--log-file=
            send all of its messages to the specified "filename.%p.%q{PerProc-EnvVariable}"
            where %p=CurrentProcID and %q{UniquePerProcEnvVariable} is used to identify a process by the unique value of its PerProc-EnvVariable.
            something like a ProcNumber maybe.
            used along with --trace-children=yes for multiprocess applications.

[Error Related options:]
These options are used by all tools that can report errors, e.g. Memcheck, but not Cachegrind.

--xml= [default: no]
            important parts of the output (e.g. tool error messages) will be in XML format rather than plain text
            the XML output will be sent to a different output channel than the plain text output
            i.e. you also must use one of --xml-fd, --xml-file or --xml-socket to specify where the XML is to be sent. 
            Note: the destination of the plain text output is controlled by --log-fd, --log-file  and --log-socket

 --xml-file=
            Specifies that Valgrind should send its XML output to the specified file. It must be used in conjunction with --xml=yes.

--demangle= [default: yes]
            Supression files contain C++ function names in their mangled form so turn off mangling for supressions.
            Turn on supressions for human readability.

--num-callers= [default: 12]
            Use this if the stack trace is not big enough, to get more call-stack depth
            specifies the maximum number of entries shown in stack traces that identify program locations.
            errors are commoned up using only the top four function locations (the place in the current function, and that of its three immediate callers).

--error-limit= [default: yes]
            When enabled, Valgrind stops reporting errors after 10,000,000 in total, or 1,000 different ones, have been seen.

--track-origins=yes
            Helps analyse the root cause of uses of uninitialised values. Otherwise you get generic messages like "Conditional jump or move depends on uninitialised value(s)"

--error-exitcode= [default: 0]
            Specifies an alternative exit code to return if Valgrind reported any errors in the run (??as in #define TestSuiteFailed 100 in cppunit testcase??)
            ... useful for using Valgrind as part of an automated test suite, since it makes it easy to detect test cases
            for which Valgrind has reported errors, just by inspecting return codes.

 --suppressions= [default: $PREFIX/lib/valgrind/default.supp]
            Specifies an extra file from which to read descriptions of errors to suppress. You may use up to 100 extra suppression files.

 --gen-suppressions= [default: no]
            When set to yes, Valgrind will pause after every error shown and print the line:
                   ---- Print suppression ? --- [Return/N/n/Y/y/C/c] ----
            Valgrind will print out a suppression for this error. You can then cut and paste it into a suppression file if you don't want to hear about the error in the future.
            This option is particularly useful with C++ programs, as it prints out the suppressions with mangled names, as required.
            You may want to common up similar ones, by adding wildcards to function names, and by using frame-level wildcards.
            The wildcarding facilities are powerful yet flexible, and with a bit of careful editing, you may be able to
            suppress a whole family of related errors with only a few suppressions.
            ... the -v option which prints out all used suppression records.

 --db-attach= [default: no]
            When enabled, Valgrind will pause after every error shown and print the line:
              ---- Attach to debugger ? --- [Return/N/n/Y/y/C/c] ----

--db-command= [default: gdb -nw %f %p]
            Specify the debugger to use with the --db-attach command. The default debugger is GDB.

--read-var-info= [default: no]
            Memcheck gives more detailed description of illegal address in its output including variable declaration line/file and thread info.

--run-libc-freeres= [default: yes]
            Force libc.so to free its private malloc's. These are normally NOT freed but fool valgrind into logging false positives.
           Note: forcing call to libc-freeres() at the program exit MAY give problems with older buggy glibc versions.

================================================================================================================

[Setting Default Options]

Note that Valgrind also reads options from three places:
   1.   The file ~/.valgrindrc
   2.   The environment variable $VALGRIND_OPTS
   3.   The file ./.valgrindrc

These are processed in the given order, before the command-line options. Options processed later override those processed earlier; for example, options in ./.valgrindrc will take precedence over those in ~/.valgrindrc.

Please note that the ./.valgrindrc file is ignored if it is marked as world writeable or not owned by the current user. This is because the ./.valgrindrc can contain options that are potentially harmful or can be used by a local attacker to execute code under your user account.

Any tool-specific options put in $VALGRIND_OPTS or the .valgrindrc files should be prefixed with the tool name and a colon. For example, if you want Memcheck to always do leak checking, you can put the following entry in ~/.valgrindrc:

--memcheck:leak-check=yes

This will be ignored if any tool other than Memcheck is run. Without the memcheck: part, this will cause problems if you select other tools that don't understand --leak-check=yes.

================================================================================================================

[Threading Support:]
+          "your program will use the native threading library, but Valgrind serialises execution so that only one (kernel) thread is running at a time.
      ...
            threaded apps run only on one CPU, even if you have a multiprocessor or multicore machine.
     ...
            Valgrind doesn't schedule the threads itself. It merely ensures that only one thread runs at once,
            using a simple locking scheme. The actual thread scheduling remains under control of the OS kernel."
+          "if you have some kind of concurrency, critical race, locking, or similar, bugs.
            In that case you might consider using the tools Helgrind and/or DRD to track them down."

+          "On Linux, Valgrind also supports direct use of the clone system call, futex and so on. clone is supported
            where either everything is shared (a thread) or nothing is shared (fork-like); partial sharing will fail.
            Again, any use of atomic instruction sequences in shared memory between processes will not work reliably."

+          "Valgrind has a fairly complete signal implementation. It should be able to cope with any POSIX-compliant use of signals."


Signal Handling:
+          "If your program dies as a result of a fatal core-dumping signal, Valgrind will generate its own core file (vgcore.NNNNN) containing your program's state."
            "the core dumps do not include all the floating point register information. In the unlikely event that Valgrind itself crashes,
            the operating system will create a core dump in the usual way."


Memory Footprint
            "Memory consumption of your program is majorly increased whilst running under Valgrind.
            This is due to the large amount of administrative information maintained behind the scenes.
            Another cause is that Valgrind dynamically translates the original executable.
            Translated, instrumented code is 12-18 times larger than the original so you can easily end up with 50+ MB of translations when running (eg) a web browser."
==================================================================================================================
Valgrind:
  • memcheck, 
  • cachegrind, 
  • multi-threaded

Valgrind Article on Memory Debugging and Profiling Howto
Valgrind Howto