Tuesday, September 28, 2010

mmap() is not the territory!! Part 2

Final code at end of debugging:

bool testFileCopyWithSharedMem(const string &srcFileName, const string &destFileName)
{
    bool isCopyOk = false;
    string diffCmd = "diff " + srcFileName + " " + destFileName;
    int retval = system(diffCmd.c_str());
    cout << "system(" << diffCmd << ") returned: " << retval << endl;
    if( retval == 0 )
    {
        isCopyOk = true;
    }
    return isCopyOk;
}

int doFileCopyWithSharedMem(const string &srcFileName, const string &destFileName, size_t sharedMemSize)
{
    //int retval = 0;

    //Map SourceFile
    errno = 0;
    int srcfd = open(srcFileName.c_str(), O_RDONLY);
    if (srcfd < 0) {
        const char * causeOfError = strerror(errno);
        cout << "open() returned:" << causeOfError << " at:" << __LINE__ << " in:" <<__FUNCTION__ << endl;
        cout << "open(" << srcFileName << ") returned: " << srcfd << endl;
        return -1;
    }
    struct stat sb;
    fstat(srcfd,&sb);
    long pageSize;
    //pageSize = sb.st_size;
    pageSize = sysconf(_SC_PAGESIZE);
    errno = 0;
    char *srcFilePtr = 0;
    srcFilePtr = (char *) mmap(0, pageSize, PROT_READ, MAP_SHARED, srcfd, 0);
    if (srcFilePtr == MAP_FAILED)
    {
        const char * causeOfError = strerror(errno);
        cout << "mmap() returned:" << causeOfError << " at:" << __LINE__ << " in:" <<__FUNCTION__ << endl;
        return -1;
    }

    //Map Dest File
    errno = 0;
    int destfd = open(destFileName.c_str(), O_RDWR|O_CREAT|O_TRUNC, 0600 );
    if (destfd < 0) {
        const char * causeOfError = strerror(errno);
        cout << "creat() returned:" << causeOfError << " at:" << __LINE__ << " in:" <<__FUNCTION__ << endl;
        cout << "creat(" << destFileName << ") returned: " << destfd << endl;
        return -1;
    }
    errno = 0;
    char *destFilePtr = 0;
    destFilePtr = (char *) mmap(0, pageSize, PROT_WRITE|PROT_READ, MAP_SHARED, destfd, 0);
    if (destFilePtr == MAP_FAILED) {
        const char * causeOfError = strerror(errno);
        cout << "mmap() returned:" << causeOfError << " at:" << __LINE__ << " in:" <<__FUNCTION__ << endl;
        return -1;
    }

    ftruncate(destfd, sb.st_size);
    cout<< "PID:" << getpid() <
    //cout << srcFilePtr << destFilePtr <
    system("cat /proc/self/maps");

    memcpy(destFilePtr, srcFilePtr, sb.st_size);
    //msync(destFilePtr, pageSize,MS_SYNC);

    munmap(srcFilePtr, pageSize);
    munmap(destFilePtr, pageSize);
    close(srcfd);
    close(destfd);
    return 0;
}
 
----------------------------------------------------------------------------
Time required:  
Debugging 1 AM to 5:40 AM. 
20 Minutes to note down these points in the blog. 
2 hrs to massage it into shape.

Interesting Links:
DevShed : development tutorials: http://www.devshed.com/
Gentoo Bug Reporting Guide: http://www.gentoo.org/doc/en/bugzilla-howto.xml 
C sample source on Gnu/Linux : http://www.c.happycodings.com/Gnu-Linux/index.html
Mmap() security bug with Null Pointers: http://blog.ksplice.com/2010/03/null-pointers-part-i/
Wiki on Chromium multi-process debugging with gdb: http://code.google.com/p/chromium/wiki/LinuxDebugging
BugReport: http://www.mail-archive.com/ubuntu-bugs@lists.ubuntu.com/msg2333427.html 


mmap is not the territory Part 1 : http://techtalkies.blogspot.com/2010/09/mmap-is-not-territory-or-mapfail-sigbus.html

Code Review Tools

Code Review Tools:
http://www.reviewboard.org/
http://code.google.com/p/rietveld/

Sample Review and Analysis of DVCS (Git/Mercurial) in Python Enhancement Proposals:
http://www.python.org/dev/peps/pep-0374/

Push your blog to google for indexing

Manually suggest your blog to google for indexing/crawling:
http://blogsearch.google.com/ping

Refer this site for further info on getting your blog indexed:
http://www.google.com/support/blogsearch/?hl=en#getlisted

mmap() is not the territory!! Part 1

(OR) how MAP_FAIL, SIGBUS,
beg to differ with mmap!!
Wrote a small program using mmap() to copy a fromFile.txt to toFile.txt using virtual memory

--------------------------------------------
Issue#1: mmap() returns MAP_FAIL
mmap() requires exactly the same flags as the filedescriptor/fd as when it was open()/creat()-ed
i.e. if you used O_RDONLY in open() then you can't mmap() it as PROT_WRITE.

chmod basics: http://www.linux.org/lessons/beginner/l14/lesson14b.html
creat/open man page: http://linux.about.com/od/commands/l/blcmdl2_open.htm
-------------------------------------------------
Issue#2:
Error Message: In gdb getting SIGBUS with error coming from inside memcpy!!

108        memcpy(destFilePtr, srcFilePtr, pageSize);
5: pageSize = 14
4: destFilePtr = 0xb7ffc000

3: srcFilePtr = 0xb7ffd000 "hello, world\n\n"
2: destfd = 6
1: srcfd = 5
(gdb) n

Program received signal SIGBUS, Bus error.
__memcpy_ia32 () at ../sysdeps/i386/i686/multiarch/../memcpy.S:75
75    ../sysdeps/i386/i686/multiarch/../memcpy.S: No such file or directory.
    in ../sysdeps/i386/i686/multiarch/../memcpy.S
 
Searching on this gave no direct answers on the error message but gave some pointers on causes.
Tried all the below tools but still getting the same error message:
0) used apt-get update and rebooted "just-in-case"
    https://help.ubuntu.com/8.04/serverguide/C/apt-get.html
1) used strace ./file-copy-vm to check the system calls were getting called properly.
   getting SIGBUS on write() ostensibly called from inside the memcpy
2) used ldd ./file-copy-vm to check that the libstdc++ - and libc.so were existent.
   They were present
3) Installed glibc-dbg and libstdc++-devel etc for debugging the library.
    No change.
Finally found the problem in an off-by-N error in memcpy(dest,src, pageSize)

Since SIGBUS comes when a page worth is allocated but size of mapping is less than pagesize. 
i.e. the accessed address is more than filesize but inside pagesize.

 So realized my mistake and changed:
        memcpy(destFilePtr, srcFilePtr, pageSize); //WRONG.
Corrected it to:
        memcpy(destFilePtr, srcFilePtr, sb.st_size); //Correct
------------------------------------------------------------------------------------------------------------------
Issue#3: Still getting SIGBUS from memcpy() AND
out of bounds>

Breakpoint 3, doFileCopyWithSharedMem (srcFileName=..., destFileName=..., sharedMemSize=8192) at ../src/file-copy-vm.cpp:104
104        cout<< "PID:" << getpid() <
4: destfd = 6
3: srcfd = 5
2: srcFilePtr = 0xb7ffd000 "hello, world\n\n"
1: destFilePtr = 0xb7ffc000
0xb7ffc00 out of bounds>

(gdb) n
PID:7136
106        system("cat /proc/self/maps");
4: destfd = 6
3: srcfd = 5
2: srcFilePtr = 0xb7ffd000 "hello, world\n\n"
1: destFilePtr = 0xb7ffc00
0xb7ffc00 out of bounds>

(gdb) shell cat /proc/7136/maps
[SNIP]
08048000-0804a000 r-xp 00000000 08:01 933913     /home/gurud/cdt-linux-tools-workspace/file-copy-vm/Debug/file-copy-vm
0804a000-0804b000 r--p 00001000 08:01 933913     /home/gurud/cdt-linux-tools-workspace/file-copy-vm/Debug/file-copy-vm
0804b000-0804c000 rw-p 00002000 08:01 933913     /home/gurud/cdt-linux-tools-workspace/file-copy-vm/Debug/file-copy-vm
0804c000-0806d000 rw-p 00000000 00:00 0          [heap]
b7fec000-b7fee000 rw-p 00000000 00:00 0
b7ffb000-b7ffc000 rw-p 00000000 00:00 0
b7ffc000-b7ffd000 rw-s 00000000 08:01 933725     /home/gurud/cdt-linux-tools-workspace/file-copy-vm/Debug/toFile.txt
b7ffd000-b7ffe000 r--s 00000000 08:01 933672     /home/gurud/cdt-linux-tools-workspace/file-copy-vm/Debug/fromFile.txt
b7ffe000-b8000000 rw-p 00000000 00:00 0
bffeb000-c0000000 rw-p 00000000 00:00 0          [stack]

Hmmm... here the destFilePtr seems to be pointing to the correct memory-mapped file i.e. toFile.txt
The destFilePtr
is pointing to out-of-bounds accesss.
Also the SIGBUG seems to suggest that I'm trying to write/access to an address in memory that's allocated but out of bounds of the empty dest file. Otherwise I'd have got a SIGSEGV if the memory had not been allocated.
 
I observed that toFile.txt is showing filesize as zero on the disk. Aha!!
 
Robert Love in his book "Linux System Programming" had mentioned files with slack-space/holes (in memory as well as disk respectively). Could this be the reason??!! 
http://www.devshed.com/c/a/BrainDump/Using-mmap-for-Advanced-File-IO/
Finally I cross-checked my code with a sample implementation for mmap() file copy program.
Found out that they used lseek() to expand the memory mapping instead of ftruncate() to expand file size with a hole instead of ftruncate().
http://www.c.happycodings.com/Gnu-Linux/code6.html

So we just need to increase the size of the dest-file using ftruncate.

(gdb) help call
Call a function in the program.
The argument is the function name and arguments, in the notation of the
current working language.  The result is printed and saved in the value
history, if it is not void.

(gdb) call ftruncate(destfd,sb.st_size)
$1 = 0

doFileCopyWithSharedMem() returned:0
system(diff ./fromFile.txt ./toFile.txt) returned: 0
testFileCopyWithSharedMem() returned:0

(gdb)

Hurray!! the unit testcase passes!! 
It's 6 AM!! Tiring but worth it!! Gotta go and sleep now

----------------------------------------------------------------------------

See Also : 
mmap is not the territory Part 2 : http://techtalkies.blogspot.in/2010/09/mmap-is-not-territory-part-2.html

Monday, September 27, 2010

Creating a CHM file from offline/mirrored HTML pages

Having too many saved/offline html folders, maybe created using Httrack like website-mirrorring software?
Storing and using these is a bit of a headache.

All you need is to convert these loose files into a compressed html file i.e. CHM file
with facilities like indexing, favorites etc.


Step 1: Install chmProcessor from http://sourceforge.net/projects/chmprocessor/. It's free to download.

Step 2: Install HTML Help Workshop if it's not already installed on your windows system

Step 3: Point chmProcessor to the HTML Help Compressor using File > Settings menu option



Step 4: Point chmProcessor to
a) the "Source File" i.e. the index.html file,
b) "Add Directory" for the directory containing the rest of the html files and
c) "Compile Help" directory to place the output .chm file


Step 5: Click "Generate" button to get the .chm file.


Gotchas: Common mistakes are to forget change the output path from one project to another.

Friday, September 24, 2010

List of useful packages for C/C++ development on Ubuntu Linux Part 2

[SOURCE-ANALYZERS]
apt-get install -y exuberant-ctags
apt-get install -y doxygen doxygen-gui
apt-get install splint #Static analyzer Lint for C programs
apt-get install colorgcc #Colorizer for GCC errors/warnings
apt-get install cppcheck #C/C++ source static analyzer
apt-get install cscope #C/C++ source code browsing/searching
apt-get install cbrowser #browser for cscope
apt-get install cflow #Displays control flow graphs for C source
apt-get install cutils #Various C source code utils - cdecl, cobfusc
apt-get install cxref #Generate HTMl doc for C source code
apt-get install global #Global search/browse C++ source code
apt-get install id-utils #Identifier database used by global for search/browse C++ source code
apt-get install synopsis #C++/Python source code introspection tool
apt-get install gccxml #GCC source code described as xml
apt-get install gobject-introspections #Extract introspection data from libraries
apt-get install explain #Helps explain system call errors after the fact
apt-get install fhist #File history, compare and merge utility
apt-get install fastdep #Generates dependency info as makefile rules for C/C++
apt-get install eresi #Reverse Engineering, instrumentation, debugging, tracing framework
apt-get install evarista #Program transformer and data-flow analyzer for binaries using ERESI
apt-get install frama-c #GUI to combine multiple analyzers for C source code
apt-get install frama-c-base #Framework to combine multiple analyzers for C source code

apt-get install gnulib #Make programs portable using C macros/assertions/declarations/definitions
apt-get install gperf gperf-ace #Generate perfect hash given input strings
apt-get install gsoap #Web service stub/skeleton generator for C/C++ code
apt-get install kodos #GUI to debug, test and view regexps
apt-get install visual-regexp #GUI in TCL to debug, test and view regexps

[BEAUTIFIER]
apt-get install astyle # C++ source code beautifier
apt-get install indent #Beautifier for C Source code
apt-get install bcpp #C++ Source beautifier
apt-get install kwstyle #Ensure source code style of many people is same as one person
apt-get install uncrustify #C++ beautifier highly configurable
apt-get install unifdef #Remove #ifdef sections from source
apt-get install universalindentgui #GUI to configure and compare multiple beautifiers esp. for C++
apt-get install xmlindent #XML beautifier

[COMPILATION]
apt-get install distcc #distributed compiling
apt-get install distcc-pump #distributed preprocessing
apt-get install ccache #compiler cacher
apt-get install distccmon-gnome #GTK+ GUI to monitor distcc
apt-get install icecc #Distributed compiling
apt-get install icecc-monitor #GUI for monitoring Distributed compiling
apt-get install boost-build #Easy Cross-platform compilation
apt-get install gdc # D language compiler
apt-get install bison++ #C++ source generator enhancement to bison
apt-get install flex
apt-get install antlr3 #create compilers/interpreters using ANTLR

[LIBRARIES]
apt-get install libasio-dev libasio-doc #Cross Platform Boost library for Async IO (network programming)
apt-get install libclthreads-dev libclthreads-doc #POSIX threads C++ library
apt-get install libcorelinux libcorelinux-examples #Converting Linux core C libs to C++ libs
apt-get install libdar-dev #Disk archiver
apt-get install uc++ uc++-doc uclibc uclibc-source #Embedded C++ with multi-threading etc.
apt-get install witty-dev #AppServer and library for C++ web-deployment

[JAVA]
apt-get install gwis #C++ wrapper class generator to call Java objects/methods.
apt-get install jaranalyzer #Dependency management utility for jar files
apt-get install jclassinfo #Reads class files to get useful info from them
apt-get install jflex javacc #Flex and Bison for Java
apt-get install junit junit-doc #Unit testing for Java
apt-get install testng #NG unit testing with extra and best-of-breed features of JUnit and NUnit
apt-get install tijmp jmp #Java memory profiler
apt-get install visualvm #Tool for remote-admin, monitoring(dumping), profiling production/dev code, bug-reporting

[DATABASE]
apt-get install sqlite-database-browser #GUI for SQLite dbs.
apt-get install sqlrelay-dev #SQLite C/C++ APIs for proxying speeding up access to N DBMS
apt-get install unixodbc-dev #Unix port of ODBC

[XML]
apt-get install xmlcopyeditor #XML util for xsd, dtd, xslt, validation and syntax highlighting
apt-get install xsdcxx #Generate C++ classes from XSDs
apt-get install xml-rpc-api2cpp #Generate C++ wrapper classes for XML-RPC API

Howto subscribe to this (and other) RSS Feeds.

RSS feeds push the blog content to you, instead of you having to check for updates to the site.
All this from within your browser or RSS feed client s/w.


You can very simply subscribe to any of the below options:
  1. The whole blog, (or)
  2. Individual blog-posts, (or)
  3. Even just the comments part of the blog.

Howto:

0) Just use the newly added gadgets at top of the blog:
   Subscribe to: Posts Comments

1) If you're using Firefox just click on the Orange Icon in your address-bar.
    And choose which folder to save your "Live Bookmark"/RSS feed into.
   Howto subscribe to RSS feeds in FireFox in seconds: http://johnbokma.com/firefox/rss-and-live-bookmarks.html


2) You can also use the below URLs:


3) Also Rss feeds to subscribe to blog comments can be found at the bottom of the blog:
        "Subscribe to: Posts (Atom)"

Reference Links:

Why write technical blog posts Part 2

Some interesting quotes (with links) from "Apprenticeship Patterns":
 
Section 6: Construct Your Curriculum
" the vast amounts of wisdom captured in the books of experienced practitioners like Jerry Weinberg, Fred Brooks, Steve McConnell, and Kent Beck cannot be replaced, not even with higher-bandwidth information. Even if you’re not a bookworm, a successful apprenticeship needs to include some books as well as time devoted to studying. You’re not in school, though. There is no assigned reading—it’s up to you to find recommendations and construct your own curriculum."

0) Reading List
          "Problem: 
                   The number of books you need to read is increasing faster than you can read them.
        Solution: 

                Maintain a Reading List to track the books you plan to read, and remember the books you’ve read."

1) Study the Classics:
"Joshua Kerievsky once asked Jerry Weinberg how he keeps up with all the books that come out. Jerry said, “Easy—I only read the great ones”
[...]
"Successful apprentices tend to focus on “long-lived books” and use the Web or experimentation to learn how the information has evolved. Dave remembers vividly the experience of reading his first classic in this field, The Psychology of Computer Programming, and marveling at how relevant the book felt, despite the stories of punch cards and room-sized computers. The wisdom captured in such classics is vital information to keep you heading in the right direction on The Long Road."

2) Dig Deeper:

"Problem:

You keep running into difficulty maintaining the code you’ve written because it turns out that the tutorials you followed cut corners and simplified complex issues. You find that your superficial knowledge of a thousand tools means you’re always floundering whenever a subtle bug arises or you have to do something that demands deep knowledge. People often accuse you of having a misleading CV because you don’t distinguish between a couple of weeks of extending an existing web service and a deep knowledge of the issues inherent in maintaining an interoperable and highly scalable enterprise system. What’s even worse is that because your knowledge is so superficial, you’re not even aware of how little you know until something or someone puts you to the test.

Solution:

Learn to dig deep into tools, technologies, and techniques. Acquire the depths of knowledge to the point that you know why things are the way they are. Depth means understanding the forces that led to a design rather than just the details of a design. For instance, it means understanding type theory (or at least the simplification offered by the typing quadrant at http://c2.com/cgi/wiki?TypingQuadrant) rather than simply parroting the things you’ve heard others say"

List of useful addons for FireFox (with links) Part 2

List of useful addons for FireFox (with links) Part 1

Note: These addons have been added on Firefox version 3.6.10

+ScribeFire Blog Editor
        Description: Edit and post your blog from inside firefox with this plugin.
                              Note: ScribeFire Next is a bit buggy i.e. publish button disappears too fast to click.
        Download: Add to Firefox


+ DownThemAll
Description:
The first and only download manager/accelerator built inside Firefox!
Download: Add to Firefox

+ Easy YouTube Video Downloader
Description:
Easiest Youtube video downloader ever, single click non-intrusive direct download buttons for FLV, 3GP, MP3, MP4, 720p HD and 1080p Full-HD qualities.
Download: Add to Firefox

+ NetVideoHunter Video Downloader
Description:
Download videos and music from video sites like Youtube.
Download: Add to Firefox

+ ImgLikeOpera
Description:
ImgLikeOpera allows load only the images that you want in Firefox browser. This extension is very useful for non broadband users...
Download: Add to Firefox
Not available for Firefox 3.6.

+ SessionManager
Description:
Session Manager saves and restores the state of all windows - either when you want it or automatically at startup and after crashes. It can also automatically save the state of open windows individually.
Download: Add to Firefox

+ SpeedDial
Description:
Direct access to your most visited websites
Download: Add to Firefox

+ WOT - Safe Browsing Tool
Description:
Would you like to know which websites you can trust? The WOT add-on is a safe surfing tool for your browser. Traffic-light rating symbols show which websites you can trust when you search, shop and surf on the Web.
Download: Add to Firefox

[BOOKMARKS]
+ TagSieve (Note: This has NOT been reviewed by Mozilla)
Description:
This is a continuation of the excellent but abandoned TagSifter extension.
Firefox lets you tag your bookmarks, but it doesn't give you a great way to browse your bookmarks by their tags.
Download:

+ Xmarks Sync (NOTE: Xmarks is SHUTTING DOWN. Don't use it.  
                     This will backup your ALL bookmarks so DON'T use for official links)
          Description:
          Xmarks is the #1 bookmarking add-on.
          Keep your bookmarks, passwords and open tabs backed up and synchronized across computers
          and browsers. Search smarter with website ratings and reviews displayed with your search results.
          Download: Add to Firefox

+ Firefox Sync by Mozilla Labs
    Description:
    Free browser add-on that lets you stay in sync with your Firefox.
    Access your history, passwords, bookmarks and even open tabs across all your devices.
    Download: Add to Firefox 

+ Bookmarks Duplicate Detector
Description:
Detects Duplicate Bookmarks when bookmarks are added and specify where is the previous URL. You can also search and delete duplicates URL already in your Bookmarks.
Download: Add to Firefox
Not available for Firefox 3.6.10

WikiSeer Keynotes
Description: WikiSeer Keynotes finds the most significant content in any English web page; shrinking the original up to 99%. It serves as a middle ground between skimming the titles and reading the entire article, providing a faster / richer reading experience.
Download: Continue to Download →

Lazarus: Form Recovery

Description: Never lose anything you type into a web form again! Lazarus securely auto-saves all forms as you type, so after a crash, server timeout, or whatever, you can go back to the form, right click, "recover form", and breathe a sigh of relief
Download:  Add to Firefox

Reframe It
Description: Commenting tool to add comments right next to Web pages without permission and share with others.
Download: Continue to Download →

DejaClick

Description: Record and bookmark your browser activities, then with a single click, replay the entire sequence all over again.
Download: Add to Firefox

FloatNotes

Description:
With this add-on you can put notes anywhere on a website. Add comments, links or even images. It can help you doing research on the web, remind you of topics you want to blog about, etc. It is simple and easy to use.
Download: Add to Firefox

Utils for Linux and Windows

Search Everything:
This is a really fast and compact file search utility for windows. It beats Google desktop hollow if all you need is to search file/directory names. Already its saved me from re-downloading stuff, finding work that I’d stored away (safely!!) somewhere and just plain discover that I’ve got more goodies on my system than I’d thought!!
Having a huge HDD (by today’s standards) 1 GB does help to keep useful stuff ready for “Search Everything”.
Some nifty features to add would have been
1) Search inside compressed files and maybe
2) Catalog generation for offline media like CDs/DVDs.

Bharat has written a nice blog post on it called "Search Everything – A wonderful utility to reduce file searches".

Recently reinstalled Windows/Ubuntu and had to reinstall all the stuff.

I was using the old blog editor in blogger and managed to lose nearly 75% of a blog.
No undo function to revert to previous version of my blog. Searched for a work-around in blogger.
Nearly 3 hours of hard-work gone down the drain.
Found these GUI utilities for Ubuntu and Windows to hold ALL copy-paste stuff from the clipboard.

1) Glipper the multiple copies clipboard manager on Ubuntu Gnome Desktop
#a) Installs Glipper the multiple copies clipboard manager into Ubuntu Gnome Desktop
apt-get install glipper
#b) Right click on the "Windows Quicklaunch Bar" and Add to Panel->Clipboard manager
#c) Click on any of the previous copies to get the copy-paste you need.

2) Nautilus elementary -> Elegant Gnome. Making the desktop look neat.
     http://gnome-look.org/content/show.php/Elegant+Gnome+Pack?content=127826
     (Found it from somewhere inside http://lifehacker.com/tag/linux/!!)

3) Also Clipboard Manager for Windows for extending the basic Windows clipboard.
      http://space.dl.sourceforge.net/project/clipman/Clipman/v1.0/ClipMan_Setup.msi
      Note: May require .net framework 2.0.50727 to run if it's not already installed.
      Easily downloadable  from this site

Why write technical blog posts Part 1


Technical blogs help to:
  • Record and speed-up the process of learning a new topic.
  • Can be used as learning devices. 
  • Provide a useful format for note-taking as you learn new things.
  • Sharing and growing knowledge with others.
  • Provide invaluable feedback necessary for your own growth.
  • Capture the essence of what you learned during research

Technical blogs should evolve from one-liners to concise researched posts.
They should act as brain-dumps of all your research.

Research is a frightfully expensive activity with loads of context building up as you search.
What happens after to all this invaluable context after you spend a week or more researching it?
It evaporates within days of research as you move on to other things.


The beginners mind full of questions is an invaluable resource.
Unfortunately it's impossible to retain as you learn more and more!!


Slowly the questions give way to answers...
Only for more questions to pop-up... with answers following slowly.
The mystery thickens fast as you try to satisfy your quest for understanding.

Researching, reading books, articles, blogs that you browse on the Net, experimenting, theorising, modelling.
All these go into the steaming cauldron of your quest....

Slowly the mists part to slowly yield insights into what makes the thing tick!!


Writing and Revising your blogs is a definite way to nurture and match this process of  growth.
The very process of writing simply and clearly forces you to seek out the core of the concept.
This is THE most valuable gift to the technical blogger - fuelling his own growth while helping others on the path.

Innocence yielding to Experience yielding (very) slowly to Mastery.



Not only does it capture the questions and answers you found.
It can very easily form the basis for helping out someone on the same path.

The Genius of Jerry Weinberg - The Psyche of Programming

Gerald (Jerry) Weinberg - The Programmer Psychologist, Writer, Systems Thinker:
          http://www.geraldmweinberg.com/Site/Home.html


This guys papers, articles and books on "Helping Smart People be Happy" are quoted on books by most of the Experts in any domain of programming.
Check out his articles/books: All books written by Gerald Weinberg on Amazon



See below for customer reviews on Amazon:

Amazon Customer Reviews/Comments on Jerry Weinberg's books:
The Classic :

Guide for Interviewers and Interviewees:

Problem Solving (useful for OnTheJob and Interviews too):

Guide for Technical Leaders:

Guide on becoming a Consultant:

Know Thyself:

Systems Thinking:

Thursday, September 23, 2010

Mindmap of my observations/experiences in "Apprenticeship Patterns Applied"

How to find an answer to (some) of your questions - in Life, the Internet and Other Forums ;-) !!!

Self-Learning


Heuristics and Problem solving


    Asking Questions on Online Forums

    + Video - Learning from Stackoverflow : http://www.youtube.com/watch?v=NWHfY_lvKIQ
















    + Video - Good Stack Overflow Citizen by Jeff Atwood : https://www.youtube.com/watch?v=HRRWDf-xB2Y















    + Article - Writing the Perfect Question by Jon Skeet : http://blogs.msmvps.com/jonskeet/2010/08/29/writing-the-perfect-question

    + Article - Answering Technical Questions Helpfully by Jon Skeet : https://codeblog.jonskeet.uk/2009/02/17/answering-technical-questions-helpfully

    + Stackoverflow Checklist for Writing a Good Question : http://meta.stackoverflow.com/questions/260648/stack-overflow-question-checklist

    ============

    1. How to Ask Questions the Smart Way (on forums)"by Eric S. Raymond : http://catb.org/%7Eesr/faqs/smart-questions.html 
    2. Stackoverflow guidelines on how to ask questions on the forum : http://stackoverflow.com/questions/ask 
    3. How to search in a smart way for previously answered questions on stackoverflow : http://stackoverflow.com/search 
    4. Questions NOT to ask : http://www.catb.org/esr/faqs/smart-questions.html#classic 
    5. Examples of Good and BAD questions : http://www.catb.org/esr/faqs/smart-questions.html#examples 
    6. How to Ask Questions (Simple English) at perlmonk : http://www.perlmonks.org/?node_id=788576 
    7. How NOT to ask a Question at perlmonk : http://www.perlmonks.org/?node=How%20%28not%29%20to%20ask%20a%20question 
    8. Before you ask the question : http://catb.org/%7Eesr/faqs/smart-questions.html#before
    9. What to Ask : http://www.catb.org/esr/faqs/smart-questions.html#asking 
    10. What to do if You Can't Get An Answer : http://www.catb.org/esr/faqs/smart-questions.html#id383575%20 
    11. How to Keep Cool and Deal With Rudeness  : http://www.catb.org/esr/faqs/smart-questions.html#keepcool 
    12. How to search for code : http://www.google.de/codesearch

    ====

    Researching

    1. Get Connected section  of  "How to do Research at MIT AI Labs"   
    2. Collected Advice on Research and Writing : http://www.cs.cmu.edu/afs/cs.cmu.edu/user/mleone/web/how-to.html  
    3. PDF- Brief advice on How to sift through research papers in your domain : http://blizzard.cs.uwaterloo.ca/keshav/home/Papers/data/07/paper-reading.pdf
    4. Videos on How to Build Your Research Career : https://www.quora.com/Are-there-good-blogs-and-books-on-how-to-do-good-scientific-research/answer/Gurudutt-Mallapur

      Tuesday, September 21, 2010

      List of useful addons for FireFox (with links) Part 1

       Addons for firefox (3.6.10)

      + Pixlr Grabber
      Description: Making screen grabs and pulling down images from the web just got a bit more easy. With the Pixlr grabber add-on you can choose to Edit, Share, Save or Copy your final grabs. You also get the possibility to right-click any image or background to edit
      Download: Add to Firefox
      + ClickCutter AutoCopy
      Description:
      Highlight a text and automatically Copy it to the clipboard.
      Based on unique Smart Autocopy technology!
      You have full control of what to copy and what not to copy when you highlight text.
      Middle button Paste option.
      Super-easy.
      Download: Add to Firefox

      + ClickCutter AutoSearch
      Description:
      Highlight text and AUTOMATICALLY SEARCH it in your favorite search site.
      Includes:
      - Automatic Copy of highlighted text (AUTOCOPY)
      - Automatic websearch for highlighted text (AUTOSEARCH).
      - Middle button Paste
      NO annoying context menus !
      Download: Add to Firefox

      + Lizardtech DjVu (Many rare ebooks are available in this format)
      Description:
      DjVu is a web-centric format and software platform for distributing documents and images. DjVu can advantageously replace PDF, PS, TIFF, JPEG, and GIF for distributing scanned documents, digital documents, or high-resolution pictures. DjVu content downloads faster, displays and renders faster, looks nicer on a screen, and consume less client resources than competing formats. DjVu images display instantly and can be smoothly zoomed and panned with no lengthy re-rendering. DjVu is used by hundreds of academic, commercial, governmental, and non-commercial web sites around the world.
      DjView4 is a new portable DjVu viewer and browser plugin.
      Download:
      http://djvu.sourceforge.net/

      + All In One Gestures
      Description:
      This extension allows you to execute common commands using mouse gestures, rocker navigation, scroll wheel navigation and page..
      Download: Add to Firefox


      + All In One Sidebar
      Description:
      AiOS lets you open various windows as sidebar panels, and quickly switch between them. So it put an end to the window chaos! In addition to bookmarks and history it opens dialogues such as downloads, add-ons and more in the sidebar.
      Download: Add to Firefox

      +Wikipedia Lookup Add-on

      Description:
      Looks up up the selected word in the Wikipedia encyclopedia in the language specified in your user preferences.
      Download: Add to Firefox

      + CHMReader
      Description:
      An extension to make firefox support Compiled HTML(.chm) file.
      Download: Add to Firefox

      + ClipMarks
      Description:
      Clipmarks lets you clip and share specific parts of articles, blog posts or anything else you read on the web. With our newly integrated Amplify service, you can post clips directly to Twitter or Facebook - or share them on clipmarks.com and more.
      Download: Add to Firefox

      + ColorfulTabs
      Description:
      The most beautiful yet the simplest add-on that makes a strong colorful appeal. Colors every tab in a different color and makes them easy to distinguish while beautifying the overall appearance of the interface. For Seamonkey & Flock. An essential.
      Download: Add to Firefox

      List of useful packages for C/C++ development on Ubuntu Linux Part 1

      Linux in a Nutshell: A superb bible for looking up Linux commands and options. http://oreilly.com/catalog/9780596154493


      Installing Development related packages on Ubuntu:
      http://software.jessies.org/salma-hayek/ubuntu-setup.html

      You can add repositories to download packages by editing this file:
      vim /etc/apt/sources.list

      For Eclipse C++ CDT installation please refer to this blog post.

      Script:
      A very useful command to record your command-line session actions for later use and replay later:
              script -a -f -t  2> timingfile  mydemo.log     #append, flush-writes, timestamp)

             scriptreplay timingfile mydemo.log    #replays with timing as per timestamps captured above
      For further info see:

           man script
           man scriptreplay

      Links:  
      http://www-users.cs.umn.edu/~skim/cs1901/script.html
      http://devdaily.com/blog/post/linux-unix/use-unix-linux-script-command-record-your-command-line-i-o

      Killing Processes on Ubuntu with 3 Finger salute(How-to-Geek)

      gpm: A command-line text mode clipboard manager with mouse integration.
      Use left mouse button to select and copy text and paste with middle mouse button.
      Super useful installation/usage howto from Gentoo: http://www.gentoo.org/doc/en/gpm.xml#doc_chap4

      gpm options: http://www.oreillynet.com/linux/cmd/cmd.csp?path=g/gpm


      apt-get install -y gpm #General Purpose Mouse server (clipboard manager for command line
      NOTE: You can set the system default packages on Ubuntu using debconf-set-selections and debconf-get-selections

      Glipper: Clipboard manager for Gnome. It retains all your text copies.
      #a) Installs Glipper the multiple copies clipboard manager into Ubuntu Gnome Desktop
      apt-get install glipper
      #b) Right click on the "Windows Quicklaunch Bar" and Add to Panel->Clipboard manager
      #c) Click on any of the previous copies to get the copy-paste you need.
      I went through the entire list of packages available and selected a few below from the Development (meta-package?) of Ubuntu (10.04.1 ubuntu desktop 32 bit)

      [COMMON-DEV]
      apt-get update #updates your packages

      apt-get install -y gcc #GNU C compiler

      apt-get install -y gcc-mingw32 #Cross Compilation
      apt-get install -y g++


      apt-get install -y make
      apt-get install -y linux-headers-`uname -r` build-essential  #C/C++ header files for building C/C++ programs 
      apt-get update eclipse eclipse-pde #Update Eclipse and plugin-dev-env for connect to p2 repository of CDT for installing the CDT

      apt-get install -y vim-gtk

      apt-get install -y c++-annotations
      apt-get install -y c-cpp-reference #Reference for C/C++
      apt-get install -y cpp-doc #C++ documentation
      apt-get install -y manpages-dev glibc-doc


      [PACKAGE-MGMT]
      apt-get install -y apt-build #GUI for apt-get
      apt-get install -y ceve #Parse linux package dependencies

      [PYTHON]
      apt-get install -y python-dev
      apt-get install -y idle2.6
      apt-get install -y cableswig #Python/Tcl wrappers for C++ code
      apt-get install -y boa-constructor #RAD tool for python/wxwidgets

      [PROFILERS]
      apt-get install -y gprof
      apt-get install -y kprof #KDE util for gprof analysis/viewing
      apt-get install -y valgrind
      apt-get install -y cachegrind
      apt-get install -y alleyoop #Valgrind related
      apt-get install -y valkyrie #GUI for valgrind
      apt-get install -y kcachegrind #KDE Gui for valgrind
      apt-get install -y sysprof #System-wide CPU profiling
      apt-get install -y systemtap systemtap-client systemtap-grapher #Collect data on a live running linux box
      apt-get install -y syslog syslog-ng #Interprocess application logging facility


      [TESTING]
      apt-get install -y libcppunit-doc libcppunit-dev  #C++ unit-testing lib

      apt-get install -y subunit #Run and Save remote test results for later use
      apt-get install -y stress #Stress test your computer with heavy load
      apt-get install -y httest #Simulate client/server actions with pattern-matching for test validation
      apt-get install -y fuzz #Stress test s/w with random inputs

      [DEBUG]
      apt-get install -y gdb gdb-doc #Debug programs
      apt-get install -y gdbserver #Remotely debug from another system where GDB is installed.
      apt-get install -y ddd #GUI for gdb
      apt-get install -y bashdb #Bash debugger
      apt-get install -y zshdb zsh-dbg #debugger and debug symbols for ZSH
      apt-get install -y gcov #Source code coverage analyzer
      apt-get install -y ggcov #GUI for gcov Source code coverage analyzer
      apt-get install -y lcov #HTML and directory-wise view of gcov output

      apt-get install -y collectd-dbg #statistics collection and monitoring daemon
      apt-get install -y wireshark wireshark-dev #Network debugging
      apt-get install tack-dbg #Diagnostic tool for correctness of terminfos.
      apt-get install -y tau tau-racy tau-examples #Multithread/multiproc tuning and analysis with GUI
      apt-get install -y leaktracer #Simple C++ leak tracer
      apt-get install -y electric-fence #malloc debugger
      apt-get install -y duma #Fork of electric-fence library with added features
      apt-get install -y winpdb #GUI debugger for Python
      apt-get install -y gtkparasite #Python interactive debugger for running GTK+ code.
      apt-get install -y happycoders-libdbg-dev #allows modern debugging paradigms for Large codebase
      apt-get install -y happycoders-libsocket-dev #
      apt-get install -y bless-hex-editor #Hex Editor
      apt-get install -y hexdiff #Visual hex difference analyzer
      apt-get install kompare #diff and merge util for files and dirs.
      apt-get install -y tkdiff #diff/merge util in TK
      apt-get install -y ht #Viewer/Editor/Analyser for all kinds of executables.
      apt-get install -y abi-compliance-checker #C++ sharedlib binary compatibility checker
      apt-get install -y tesq #Decoding of terminal escape sequences used by Unix Terminals