Wednesday, May 23, 2012

Permissions on Isbrae

So, another thing that always comes back to haunt people (myself included) is permissions.  You copied something to /bigtmp (see other posts on that topic), and told someone where to find it, but they came back and said they couldn't copy it, read it, or whatever.  The problem is probably permissions.

Unix (linux and Mac os X are unix variants) file permissions work on three levels:  there are separate permissions for the owner of the file, the group to whom the owner belongs, and the entire world, which means everyone else on the system.  The owner of a file is the one who sets these permissions.

To see the permissions on a file or directory, you need to add the "-l" flag to the common "ls" command for listing files (for "long listing"):

[bo@isbrae ~]$ ls -l /bigtmp/
total 193860
-rw------- 1 bo    glaciology   2720463 May 23 13:26 katie_bos_radar.mat
-rw-r--r-- 1 bo    glaciology      1573 May 23 13:26 katie_density_core.txt
-rw------- 1 bo    glaciology      6385 May 23 13:14 summit_optimization.m

drwxr-xr-x 2 bo    glaciology      4096 May 23 14:30 temp_dir
-rwxrwxrwx 1 bo    glaciology         0 May 23 14:32 all_permissions_file

Here we see a few files I copied to /bigtmp for Laura to check out.  Each line is for a file.  The first thing you see in the line is 10 characters, which contain the file type and permissions.  The first character determines file type- it's a '-' for a regular file, and 'd' for a directory.  There are other characters you might see in that spot, but don't worry about that for now.  Next there are three sets of three characters, indicating the file permissions for, again, the owner, group, and world.  Directly after these permissions characters the owner is listed (in this case me, 'bo'), the group, the file size in bytes, date and time last modified, and finally the file name.

So lets look at what's here.  I've created an empty file (file size 0 bytes) called "all_permissions_file" to illustrate the permissions possibilities.  Each of the three groups gets to read, write, or execute (rwx) this file.  In general, it's not a good idea to give anyone write permissions on a file unless you are sure you want to, because it'd make it easy for people to accidentally delete your work.  So note that for all the other files and directories in this example, the owner is the only one with write permission.

Now take a look at the directory "temp_dir".  All users have read and execute permission, but only the owner has write permission.  This means that only the owner can put things in this directory, and only the owner can rename or delete the directory.  Execute permissions in this case are important, because though others have read permission, to list the contents of a directory requires execute permission.

Next look at a file that is all set to be shared, "katie_density_core.txt".  This file has read permissions for everyone, and write permissions for the owner.  So now Laura can copy this file to her space, giving her ownership over it and thus control.  

There are two files, though, that will thwart any efforts to view, copy, move, delete, rename, or otherwise mess with them, "katie_bos_radar.mat" and "summit_optimization.m".  Only the owner has any permissions over them.  Note that as owner you can even withdraw these permissions from yourself, though it would make the file less useful (one exception is that if you want to keep yourself from accidentally deleting a file you could take away your own write permissions).  But now I, as owner, need to change the permissions so that Laura can copy these files too.  To do this you use the command "chmod" (think "change mode").  There's a lot to this command, and I'm only going to give you the simplest way to use it- give all users read permissions, using the "a+r" flag:

[bo@isbrae ~]$ chmod a+r /bigtmp/katie_bos_radar.mat 
[bo@isbrae ~]$ chmod a+r /bigtmp/summit_optimization.m 

Now look to see that everything is as it should be, 

[bo@isbrae ~]$ ls -l /bigtmp/
total 193860
-rw-r--r-- 1 bo    glaciology   2720463 May 23 13:26 katie_bos_radar.mat
-rw-r--r-- 1 bo    glaciology      1573 May 23 13:26 katie_density_core.txt
-rw-r--r-- 1 bo    glaciology      6385 May 23 13:14 summit_optimization.m

drwxr-xr-x 2 bo    glaciology      4096 May 23 14:30 temp_dir

-rwxrwxrwx 1 bo    glaciology         0 May 23 14:32 all_permissions_file

[bo@isbrae ~]$ 

And you can now safely tell your colleague that the file is available.  Note that if there is a problem with permissions, only the owner can fix it, although as the "root" user, a system administrator can also come in and fix things as well.

Hope this helps!

Wednesday, May 16, 2012

Logging into isbrae

So, someone's told you to find something on isbrae.  Huh?

isbrae is our group linux server- it's got many CPU cores, and more importantly, many Tb of data storage space.  So it's a good place to store our data, and if you want to, since it runs matlab, python, and has most of the popular compilers and many graphical visualization packages you can do all of your computing there as well. 

But first you have to get there.  From a mac, open the "terminal" application, which is under Applications -> Utilities.  From Windows, you can use a freeware program called "Putty". 

Once you open the terminal, you can type your command to ssh or "secure shell" into isbrae:

HawleyMBP:~ bo$ ssh
Last login: Tue May 15 09:57:34 2012 from
[bo@isbrae ~]$

Note that in this case, my username on isbrae is 'bo' so you will need to replace this command with "ssh"  You will be asked for your password, which you will have set when you got your account. 

If you want to run programs on isbrae that contain graphical output (like matlab running in graphics mode, for example), you need to "pipe" the graphical display from isbrae to your local machine.  This can be done easily by using the "-X" switch on ssh:

HawleyMBP:~ bo$ ssh -X
Last login: Wed May 16 10:36:46 2012 from
[bo@isbrae ~]$

It's got to be a capital X, not lowercase, because lowercase means explicitly "do _not_ forward graphics". 

Note that you need to have "X-windows" installed on your local machine.  For macs, this comes with the developer tools and extras and may not have been installed by default, but should be on your install CD.  For windows, you will need an "X emulator", and will need more help than this blog post can provide...

Graphical SCP to and from /bigtmp/

So, although I prefer to use the command line, I realize that for some, pointing and clicking is an easier way to do things.  It is for these folks that this post is written.
SCP is simply a protocol for copying files securely from one networked machine to another.  The command line is to my mind the simplest way to execute this protocol, but there are other ways as well.

For those who wish to use a point-and-click interface, there's fugu for the mac:

or FileZilla for mac, windows, or linux:

Since I don't personally use either of these much (though I have used them occasionally in the past), I won't go into a tutorial here- but as they are both open source projects, google will surely help if you need guidance.  Happy clicking!

Tuesday, May 15, 2012

Moving things to and from /bigtmp/

/bigtmp/ is our temporary swap directory on Isbrae.  There are lots of ways to get thigns to and from /bigtmp, depending on what platform you use, where you are (whether you are hard-wired to the Fairchild subnet), and your personal preference.

First off, take a look at bigtmp by logging into isbrae, to see what's there:  Open a terminal (I'm on HawleyMBP, my macbook pro) and at your prompt:

HawleyMBP:~ bo$ ssh bo@isbrae
Last login: Mon May 14 17:38:03 2012 from
[bo@isbrae ~]$ ls /bigtmp/
[bo@isbrae ~]$

Looks like there's nothing there now- which is not surprising, since bigtmp is wiped clean every weekend (otherwise it wouldn't be very tmp, would it?).

So lets find a file we'd like to put up onto isbrae.  I'll use a matlab file I created for Blythe:

HawleyMBP:~ bo$ ls -lh for_blythe.mat
-rw-r--r--  1 bo  glaciology   458M May 15 09:53 for_blythe.mat
HawleyMBP:~ bo$

Note that this file would never be possible to email, as it's 485Mb! 

So, to copy is up to isbrae, I use "secure copy" or scp:

HawleyMBP:~ bo$ scp for_blythe.mat bo@isbrae:/bigtmp/
for_blythe.mat                                100%  458MB  11.5MB/s   00:40   
HawleyMBP:~ bo$

And then log in and check that the permissions are correct:

HawleyMBP:~ bo$ ssh isbrae
Last login: Tue May 15 09:57:20 2012 from
[bo@isbrae ~]$ ls -lh /bigtmp/
total 459M
-rw-r--r-- 1 bo glaciology 459M May 15 09:55 for_blythe.mat
[bo@isbrae ~]$

and those last two 'r's in the permissions line show that both the workgroup (glaciology) and in fact any user on isbrae has read permissions.  So you can send your colleague the path, and he or she should be able to pick it up!  Next time- doing it in another way!