Online vim tutor - web based

For anyone who wanted to have the most very basic of commands to use in VIM in order to break into the command line world, there is a very good online tutorial which is nice for learning vim from a system that doesn't have it installed.  The online version has a specific sub-set of commands which a person would learn from vimtutor, but it is a good start and offers step through video side-by-side.

This might spark the question of "why".  The most impressive first reason to learn vim if you aren't using a linux system would be to log into a system as a user to make a text file in a hidden little corner of their home network.  I have such a server in the house where there is the shared printer, file shares, media and all home servers are run.

Obviously, this is the gateway drug to wanting to learn more, to run vimtutor once logged into that system and then to go online and look up every key combination available.  I still stick to my decision to use vim.  I know the vim vs. emacs war is long and the history trails with the text file bodies of the innocent victims.  I do personally think that emacs is probably much more extensible, but it is default included in 0 popular operating systems.  While every Unix and Linux kernel have vi (vim's grandfather) included and you will never find a system without at least vi.

--
CafeNinja

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

Home Server deployment - Part 1

So I got a home server (nettop system dual core atom).  I'm migrating my current server applications from the wife's desktop (always on) system to the new low power consumption system.

Here is what I ordered:
Shuttle XS35GT Barebones Nettop (No OS) (1.66GHz, 2GB, 500GB, No Monitor)

So I have already migrated dnsmasq (copy config and hosts files), ddclient to update the "way home", nzb grabber and samba which is the method I've gone to since my mythtv setup went bonkers in my 10.10 upgrade.

I'm running Ubuntu 10.04 LTS server to ensure stability.  I'm hoping it should prevent issues with stability.

I've got on the list to re-deploy mythtv, asterisk, openarena-server, a LAMP stack with some CMS system that will let the wife have/manage a website, statusnet and if possible I'll try to set up a server to receive the mozilla sync or xmarks in such a way that my "personal cloud" should be close.

I'm making a conscious choice to not migrate email as that system needs to be higher availability and accessibility than my home dsl and system can manage.

I'll try to chronicle my tales here.  So far it's been pretty easy to migrate servers by installing the server and then moving the configuration file.  The biggest issue so far has been me who forgets network orientation (one IP vs. another) or file permissions.  Outside of that, it's been pretty easy.

--
CafeNinja

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

Let's start talking about keys

I have received quite a few requests after the last Tin Foil Hat Show asking for gpg keys to be shared so that webs of trust could be built. While I won't be able to organize a key sharing party like the one described in "Little Brother" by Cory Doctorow, I do believe in the security and trust levels that are supported by GPG keys.

I'd like to offer my key for the Tin Foil Hat Show as the contact point for key sharing. I would be happy to sign, read or verify anyone's key if you wish to share it. My key is "0x5110932D" and you should be able to import tinfoilhatshow@gmail.com from hkp://pgp.mit.edu:11371 or hkp://keyserver.ubuntu.com:11371 as I have uploaded them there.

If you have created a key based on Tin Foil Hat Show, make sure you export them to a keyserver so that it can be validated as a real key. I look forward to anyone who would like to test their keys or signatures out writing to tinfoilhatshow@gmail.com.

--

CafeNinja

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

CLI - Vim and Awk

Vim and awk.  It is great to make a list, then modify that list and save it out.  These types of tools are why using any text editor on Linux is way more powerful than on the competition's operating system.  Just like with most solutions in Linux is that the tools are probably too many and do too much so the first steps are overwhelming with choices.

While a use case that doesn't happen every day here is the quick geek-mode of explaining a recent use I had.   Situation: home server runs dnsmasq (dns and dhcp server) for the home network.  I needed to ping assigned IP's for a monitoring script.  This is the geek-mode of explaining what I did.  With the resulting screenshot.

$vim fake-hosts   

This makes the new empty file.

:r !cat /etc/hosts

Fills the file with the current hosts file contents.  I've cleaned up comments and lines that won't change by using "dd" to delete the self referring IP's and lines that start with "#" since they aren't really IP addresses

%!awk '{print "ping "$1}'

This takes the IP column and prints it as the second column while creating a new first column with only the word 'ping' and notice that there is the space included inside the double quotes.

Remember, if you make a mistake with one of these that while in the CMD mode of vim the keyboard command "u" does undo just as you might expect it should. 

This process works differently than a search and replace where you need to change the subnet, it would make since to just do a search and replace of "168.0." -> "168.2." which would replace only that value.

This process works if you need to do transitive work on a text file that isn't a csv format.  Note that with this tool it is easy to take a list of results and make a csv with something like  %!awk '{print $1 ", " $2 ", " $3}'  and the 3 column file turns into a csv.  But wait, the original is correct, but in the wrong order, make your corrected csv with the following -> %!awk '{print $2 ", " $3 ", " $1}' with awesome results.

Play with it, have fun.

--
CafeNinja

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

Fun CLI uses

So if nothing we have learned that command line interfaces are not for novice users, very efficient, loved by CafeNinja and usually very boring and never any fun.

Well that isn't quite true.  I follow one my my favorite sites commandlinefu.com and have in the last few weeks seen some great fun uses of commands that can make anyone look to be a command line ninja with no further training.

The first command that I noticed makes use of a ncurses dialog box.  The command reads like this:

"dialog --gauge "Processing..." 6 40 36"  This defines a progress par that has reached 36 percent with the other numbers being the measure of the progress box.  This doesn't "do" anything, it is stuck in this way and uses no resources, but can make you look quite busy "processing" something.  It looks like the image I've put here.  You make it go away by just pressing Ctrl+C.


The next one make the screen just spew gibberish from the random number generator.  The only "problem" with this one is that it can drive even new processors to spike 100%, even worse than a run away flash page.  I added the "timelimit" command portion to prevent this running on a system that I'm not watching closely.  The command itself is 'cat /dev/urandom | hexdump -d |grep "25"', and will run crazy on your cpu.  My version is : 'timelimit -t120 cat /dev/urandom | hexdump -d |grep "25"'  This looks like this:


There are other commands that can actually make the progress bar count up like "for i in `seq 0 100`;do timeout 6 dialog --gauge "Install..." 6 40 "$i";done"  which like the first command is only making a dialog box that ins't really doing anything besides counting.  Look for your favorite and post them back here.  I bet we can make a small collection that if launched and run in an individual Terminator tab could always make you look diligent and beat the crap outta any "Boss key".

--
CafeNinja

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

App Review (Update) : tty-clock - working on Ubuntu

Previously I mentioned an app called tty-clock.  I really do like it, but on Ubuntu it didn't quite compile nice so I thought I would give the quick "howto" get it working.

1.) Grab source code and unpack
2.) install the dependencies with "sudo apt-get install ncurses-dev ncurses-runtime ncurses-term"
3**.) Optional--Open the Makefile in the source code folder and change the $INSTALLPATH=/usr/bin/
4.) Terminal in the source code directory and do "make" (should have no errors) and then "sudo make install".

After that the tty-clock should be accessible.   For your information I use the flags to display seconds, center the time and set the color to blue.   Looks like "tty-clock -s -c -C 6" and I hope that might help you get it working on your Ubuntu system.   I've added a screen grab of my clock so you can see what the command does output.

--
CafeNinja

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

Command Line Time - tty-clock

I was playing around with Arch Linux (another blog post soon to follow) and I found a neat little cli application.  I should say "cute" instead of neat.  I should also put this application find in the context in which I use it.

The find is tty-clock, it is a simple digital clock with only a few options.  In my daily pattern I have terminator open with multiple screen sessions to multiple servers.  In each of the screen sessions I'm using the byobu (formerly known as screen-profiles) to give myself a "taskbar" for the screen sessions.  This app, is then just one horizontal and vertical split in a screen filled with terminator with many tabs and splits already.  and it just means that without squinting, I can actually see the time.

This is what tty-clock looks like in action.  With just large Atari 2600 generation 8-bit color and date display centered in the open terminator split it is simply understated and serves it's purpose.  ->

I did also find binclock.py which is an executable script and while it serves the same purpose is the binary clock version.  After downloading, just set the python script to executable and put it in your path (/usr/bin/ or so) and run it. So yet one more split with even more time telling fun.






I do want to give all the credit to the links I found these gems at, I was for the most part cruising and looking for Arch Linux stuff and I came across this stuff.  So I found these apps by looking through K.Mandla’s blog, and Yu-Jie Lin's feedburner got me some other stuff.  Seems they are cli junkies like me.

--
CafeNinja

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

sc - command line spreadsheet

No surprise to folks who read my blog, I really dig on command line apps.  Well, after another visit to my favorite web based list for cli apps (thanks Jared Lee) and started to play with sc (available even in Lucid 10.04 repos).  This app is for command line spreadsheets.

Command line spreadsheets, well, yes.  I actually have a few use cases where a sqlite database would be way over the top but there is some math I would prefer not to do by hand.  That's what spreadsheets are good for.  sc comes to the rescue.

While I'll confess there is no easy import/export via csv or some other stuff, it is very versatile and does offer a text export which calculates the formulas and shows the layout as if you were using the app.  Let me show you a few quick shots.

First, using the app:

Second,  the format of the file while being used by sc, bear in mind this is also clear text:

Lastly, this is what the text export of the same document looks like:

I hope this might give you some ideas on how to use this quick, simple and easy to use program in your personal workflow.

--
CafeNinja

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

Discovered Nugget - nmon

 I have to confess that I picked up on this little command line gem as a result of my subscription to a paper magazine.   Linux Magazine had a feature written by Charly Kühnast describing the application "nmon".

This app was a simple "sudo apt-get install nmon" away and was in my Ubuntu 10.04 repos with no further setting changes required.  The application does need to be run in a terminal.  It does not require root permissions and has a great many monitors that are available for viewing in both a real-time and a collection mode depending on how geek your statistic and performance needs are.

I have a screen shot here of the application running in a tall 1/2 wide screen with just a few (not all) of the monitors activated.

I was quite impressed with the monitors/sensors available and if I read correctly there is even a "plugin" system by which monitoring scripts might be plugged in to display monitors that are not predefined.

In my set up I have activated all except memory and processes and run the "htop" command in the a terminal adjacent which then shows almost all possible performance real-time data available in one manageable and comprehensible screen.

Enjoy this app and add it to your list of favorites to use while embracing the command line.

--
CafeNinja



This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

Dnsmasq for your personal domain and access list needs

With the addition of the dnsmasq server to my LAN (preferably wired) based server, I am able to manage dhcp myself.  To what benefit you may ask, I'm glad you asked.  First is that I can actually set the IP of computers connecting to my network, and group as needed/preferred.  I can also use MAC address to authenticate, so hardware address filtering is not limited by small cheap routing equipment (I discovered a limit of 30 known hardware addresses to put in my wifi router).

So the most minor feature is that when I connect via ssh to my server from the internet back to home, I can use proper hostnames which I know will always resolve to those computers.  No more ssh'ing to one IP after another, just use the name of the computer and if connected will always work.

Simple "sudo apt-get install dnsmasq" and edit the dnsmasq.conf and hosts files.  It is heavy with comments in language which is moderately clear.  There are many guides on the open internet to follow if you have questions.  Turn it on for the safe host name resolution of your internal network.

--
CafeNinja

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

Mutt - the cool kid's RC file

 I've gotten quite a few recent requests in regards to my muttrc file.  I blame notKlaatu for that.  For those who don't know, it's the file that sets all the preferences in the most awesome text based email reader, mutt.

So I'll keep the post short since the file speaks for itself.  It has comments in it.  As a disclaimer this one is being used with mutt 1.5.20 (2009-06-14) patched with the sidebar patch (see previous posts).

Get a zip of my Gmail prepared muttrc file here.

Enjoy everyone.  Tell your friends to come by and have a peek.

--
CafeNinja

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

Backup! Backup! Restore!

I'm sure many of my fans probably think that I only use the command line.  While true that I do use command line considerably.  I also use gui tools.  As I just set up yet another virtual machine in Virtual Box, I thougth I might explain how I add the world of text to gui.

So, I've written about my gpg encrypted password file.  I leave it in my dropbox account as well as a the gnupg.vim plugin.  Of course, dropbox is primarily a gui application, which has a command line element that can be used after the gui has been launched.  So the first thing I do is set that up my password file.  Then also from dropbox I have bashrc and bash_aliases.  I soft link those to the home directory as well as the bin/ folder where I dabble in scripting.

After my last install, I started making my list of apps that I install on my Ubuntu systems knowing they are in the repos.  Basically I just cat that file and paste and wait. 

While that's going, I get the firefox up and running where I add my xmarks extension and run it.   This brings by browser up to speed more or less.  Once my chromium is installed, I just have it import the settings from firefox (google chrome the same).  Again, that's a matter of a gui application that I do rely on and use on many different systems, even cross platform.

This puts me in very familiar teritory and within a short time after a fresh install, I have my bash shell restored with my comfort zone of aliases, my browser set up with bookmarks and the odds and ends that I keep in my Dropbox.   The single behaviour modification that I keep on is making sure that anything I "might need later" I put in my dropbox or my Ubuntu One (gui).  Included in that would be the most recent .deb for skype, google chrome beta, and anything else where just having it arrive in the dropbox/Ubuntu One sync keeps me from having to hunt it down online.

This kind of file syncing actually has made system restore or first set up quite easy.  I have done this a few times and now it feels quite "normal" and I have no worries at all if any one (or truth all) of my systems were to die, I could still recover in short order.  Working directly out of my backup system/service keeps me always on the spot should anything go wrong.

--
CafeNinja

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

Jared Bernard - CLI user

 I heard on HPR episode a short talk given by Jared Bernard at the Utah Open source conference on his use of the command line.  I had absolutely no disagreement with what he said and was amazingly impressed with his list of command line application alternatives.

Really, this blog post is to promote Jared's site.  I think it's a great resource and found it educational if for nothing other than the list of CLI Applications.

If you would like to hear what he had to see please listen to HPR Eps 0518 "Live without a GUI" and visit his site.

Honestly, with the long list of applications he has listed there, I might be busy for quite a while reviewing the applications he has on the list.  Thanks Jared, and keep up the great work.

--
CafeNinja

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

Steganography - Some new fun

After reading an article in Linux Magazine, I've been playing with the steghide program available on most linux distros and popular repos.  I am quite impressed with this programs ability to insert quite a bit of information into an image or wav file.

The word steganography is of Greek origin and means "concealed writing" (citation wikipedia).  Really, as a word it is just describing covert messaging.  It offers no context in which it must occur or any kind of standard.  This technique is designed to put the message in a quite public place, kind of like the symbology clues you might have heard about in "The DaVinci Code".  The concept of security through obscurity is primary here.

The program recodes color codes in a jpg file to replace a shade of color with another and then uses the digit difference to insert the message.  The steghide program actually takes the image and the message and shuffles them together on the level of the data.  The receiver would NEED to know to look for the message since there is no visible indicator of a hidden message.

The application steghide is a command line tool.  There is a gui tool called steg-gui, but I didn't feel like compiling it, and since not in my repos I didn't use it yet.  So I won't offer any opinion of it here, maybe in the future if it can manage to be included in the repos.

As another example.  The image of Bettsy above, is also a steganograph.  Feel free to have a go and treat the image like a captcha.  After steghide is installed the following will produce the message that is this blog post, give it a try.

wget https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8lzJH8uunbgYj61D3nW8Exqt_2XR5lVu4SGFbx53bOXXHasWOPduf5cqEgyGSrERsvep3eWKHFX2k1Evq06Uu9otnaVpBe32LznDwxhV3lTFBcLdOy7cFfmhiK1PwfhhTfO0rgGVq6v91/s320/steg-post.jpg && steghide --extract -sf steg-post.jpg

Enjoy,
--

CafeNinja

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

Inx 1.1 review

After listening to TiT Radio Episode #15, I found the link that was suggested by Azimuth.  This is really, really cool if you want to get some serious command line goodness.

The distro is called inx 1.1  and it is based on ubuntu 8.04 LTS.  It is also only a live distro, currently there are no simple install methods.  There is a Virtualbox image that you can download and just run without any burning/rebooting/hacking involved.  I am going to request that any and all geeks who would like to see this project continue to send feedback to the project.  I think it's fantastic and think it would be a great idea for a meta-package to make sure all the cli/ncurses apps that are used in his menu scripts are there, and then have this as an option from gdm to boot into (saving even more resources).

I am VERY taken with this.  There are just enough tools to do 98% of what people do with their computers and brings it down to the lowest common denominator.  I'm going to be trying to send as much feedback as possible to try and help those guys prepare for a new release using 10.4 when it comes out since there have been some dramatic improvements in the package selection since 8.04 (like screen-profiles a.k.a byobu).  Here is an example of some of the cli goodness you get with this tiny and non-resouce intensive distro:

I will be following this distro closely as it will help me also to have multiple VM machines which actually have some applications in them and do more than reply to ping when testing network apps on my host and home network.  If you look close at that screenshot the RAM footprint was almost 50mb!  Again, that is loaded with apps that are useful if you only take some time to make yourself familiar with them.

Enjoy CLI Geeks, and even if you don't use the whole distro, dig in there and check you the apps that come with the distro to see what you can add your command line arsenal.

--
CafeNinja

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

Dropbox on a Headless Server

I know there are a few guides for Dropbox on a headless server.  I noticed that my experience wasn't 100% as advertised, so I wanted to post here on what my process was (repeated on 3 servers for accuracy)

First the requisites for this activity.  You must have a ssh account and my advice is to make sure you have the same disk space free on the server that you are using in your Dropbox account.  For the free version, that means 2Gb of disk space.

Next I need to install the application, this is done on the server account.  I had in all 3 cases to launch the "nautilus" application one time.  So I connect with the "ssh -X" for allowing X11 export over the connection.  Once connected the next thing was to get the dbcli.py script from the Dropbox Wiki.  On the server I used:
   wget http://dl.getdropbox.com/u/43645/dbcli.py
   chmod 755 dbcli.py
This gives the application in the home dir with executable permissions.  Now run the script with:
   python dbcli.py install     <--- mine was x86 the other is x86_64
This step will download the tarball and start to set things up.  Mine hung after it downloaded everything and put it in the right place.  I then escaped with ctrl+c.  I then launched nautilus with:
   nautilus
This brought up the view of the home folder and then the splash screen for setting up the Dropbox account.  I entered my account details and let the folder sync the first time.  Once done, I closed nautilus and disconnected from the server to forcefully break the 2nd Dropbox icon it put in my computer's systray.  I then reconnected WITHOUT x11 forwarding and launched the daemon in the background with:
  ./.dropbox-dist/dropboxd &
This should fire up the daemon to run in the background.   After that you can check the status of the daemon with the dbcli.py script with:
  python dbcli.py status
That should report "up to date" as long as it is running and healty.

You might need to restart the daemon if the server machine reboots.  But for my headless servers that doesn't happen very often.  I have seen this work perfectly with the syncing that Dropbox does and works just as expected and seen with any other desktop machine.  I really enjoy this idea since there is a disk space use limit built into the account.  Also, this provides a truely off-site backup storage to ensure against any disaster recovery issues.  So with my last post about using gpg to encrypt files and edit them, it means that even if the server security is compromised, they won't get any data from my files unless they have a supercomputer and 20 years :)

--
CafeNinja

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

vim + gpg + dropbox = secure passwords

Finally accomplished a multi-system synchronized encrypted password file.  Let me tell the short and then the long version.

The short version is
1.) gpg encrypt a text file
2.) store it in a shared dropbox folder
3.) add vim gnugpg plugin.

The long version of the story is that Dropbox has added a functionality to share a folder with other Dropbox users.  I have a Dropbox account for each operating system that I use (3).  So I can modify on one machine and it will get pushed to all the other machines.  I also found the dbcli.py script which lets me run dropbox on a headless machine.

I already have 2048 bit pgp keys created.  I used that key to encrypt a text file to myself.  The file I had was a text file that I was using to record my passwords.  With the command "gpg --encrypt filename"  will produce an encrypted copy with the *.gpg extension.  This new file I placed in the shared dropbox folder and created a symbolic link to it in my home directory.

Then after creating the folder ~/.vim/plugin and moving the gnupg.vim into that folder I was able to open and edit the gpg encrypted file after typing my passphase for the key.  This extension turns off the auto-backup feature of vim to prevent a clear text copy from being anywhere but ram.  Once writes to the file are made, it will re-synchronize with the other computers automatically.

With this new process I am able to view, search and edit my password file even on a mac (with vim and gpg installed).  Of note, I also have a Truecrypt volume in this same shared folder and once a month I do a plain text export into the hidden volume of a Trucrypt drive.  This allows me additional access even if I do not have the permissions to install all the needed components.

Now I can use strong secure and unique passwords for every online service I have.  I even took the opportunity to generate a few passwords and keep them in the password file directly in the case I needed to update a login without command line access to pwgen.

--
CafeNinja

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

Password Generation for security

Security and safe passwords are very important in today's online world.  I used to have unique passwords for "levels" of security.  Realizing that if some service from low level was compromised that access to the next level would be easy.  I realized that the best way would be a strong secure password for each online service was the best way to go.  This way there is no way to "escalate" privilege level from one online service to another.

Since I already have a method for recording my passwords, which I'll get into during another post, I though I'd share the command that I use to create new passwords.  I use the package "pwgen".  The output of this command as quoted by Bessy produces quite a few unique passwords which satisfy most secure password definitions.  Most are produced with a portion of it that can be pronounced as a word, which make them in the realm of possibility to actually remember.

Output looks like:

cafeninja@vegeta:~$ pwgen --alt-phonics --capitalize 9 10
joo0kai2Y loh6Chi6y Chei4pei8 AiNoch2ua eeXeeF8Ee veeh3Eer4 Vah2fee8v ve4uegh7E Oa6chahmo Ish2voc2c

Using this tool I can make secure passwords for any account online.  I recently used this command repeatedly as I converted my multitude of online passwords from levels (5 different passwords) to a single password for each.

--
CafeNinja

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

Karmic testing - Easier PPA additional and install

So I read on the Ubuntu Geek blog, that in Ubuntu 9.10 Karmic there is a new command for adding a ppa repo that will update the sources.list and import the gpg key.  So, clever me, thought of a way to make a few nice aliases and run this all together.  Let me demonstrate

1.) add the repo.  
sudo add-apt-repository
2.) update the repos
sudo apt-get -qq update
3.) install the application
sudo apt-get install

So I made aliases for those three.
alias repo='sudo apt-get-repository'
alias update='sudo apt-get -qq update'
alias ins='sudo apt-get install'

And using a different example from Ubuntu Geek, the command I give looks like:
"repo ppa:jonabeck && update && ins ifuse"

This single, short command will update the sources.list, update the repos and install the ifuse package with only the normal prompts if they apply.  Please see the manpage for each of these commands as well as alias in the case that you want to learn more about each.

 --
CafeNinja

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.