Warriors

Recently, Ratko Mladić (aka "The Butcher of Bosnia") was captured, 16 years after the Srebrenica massacre. I got interested in the details of the Bosnian civil war after watching the very depressing BBC TV-series "Warriors" many years ago, which deals with events a few years before Srebrenica.

The TV-series illustrate the impotence of the UNPROFOR peacekeeping force during the ethnic cleaning of civil Bosniaks in the Lašva Valley by Croatian nationalists near Vitez in 1992/93. The movie and its characters are fictional, however based on interviews with British UNPROFOR soldiers who served there. The forced passiveness of the soldiers as the crimes took place and the resulting trauma is one of the key themes in this movie.

The TV-series was aired in 1999 and is in two episodes. It can be found on YouTube, divided into 10 minute bits. The start may be a bit slow, but don't let that stop you:

http://www.youtube.com/watch?v=9T_VBa9krlk


(use link above if embedding below fails)





In 1995 after the second Markale massacre, the Croatian Operation Storm along with massive airstrikes by NATO (Operation Deliberate Force) led to the Dayton agreement which ended the war.

S.T.A.L.K.E.R.

In an earlier post I mentioned how the movie by Andrei Tarkovsky utilized the urban exploration concept in his "Stalker" movie long before wasteland tourism became in vogue.

The S.T.A.L.K.E.R. computer game series by Ukranian GSC are based on Tarkovskys movie. The Games are centered around the Chernobyl area, naturally enough, which bears several similarities to the "zone" concept of the orginal movie, even if the nuclear accident happend several years after the movie was released.

In fact, this games series was my main reason to get a new computer with a decent graphics card, I simply had to experience it! And I was truly spellbound by them. Ok, indeed the first two games, Shadow of Chernobyl and the prequel Clear Sky, had plenty of bad bugs. Call of Pripyat however improved a lot on the technical side. Also, as all first person shooters, they carry a solid amount of violence, but that is not really the point here. The atmosphere of the games along with the wonderful graphics is simply staggering.

The developers put a lot of care and detail in the graphics development, it took me some time to realize that most buildings and sites in the game actually exist in reality.

For this post I made a ton of screen shots, and compared some of them with real pictures I could find via google maps and panoramio.
(Click on images for a full scale version, most of them are very dark, but improve when viewed in full size.)

A model of yanov railway station at noon and dusk.

Yanov railway station is located south of Pripyat.
http://www.panoramio.com/photo/20352657

Yanov station in reality. 

In the western part of Pripyat there is the "Jupiter" factory, as seen with google maps:

View Larger Map

Below is a screen shot taken while inside the factory. The amount of details in the shadows rendered by the sun really demonstrates how computer technology has accelerated in the last 10 years. When I still was a little kid, a super computer might take days to render such an image via raytracing. Now they are generated faster than 50 times a second, and the CPU still has time to do the physics engine, artificial intelligence of non-player characters etc... Whee, I feel very old now!

The "Jupiter" factory in real life.

See http://www.panoramio.com/photo/21305364

I was astonished how close the game developers stuck to the real layout of Pripyat city. The hospital, kindergarten, shops, movie theater, landmarks and building blocks are located at the correct places.

"Univermag", Soviet time supermarket.

Same place in reality.

http://www.panoramio.com/photo/27682851

What really adds a lot of quallity to the game is the fantastic lighting engine. The movement of the sun with all the resulting shadows are rendered on the fly. Different weather conditions and positions of the sun, dims or increases contrast of the shadows.

Awesome lighting engine! And walls half painted in blue-green colors, very Soviet Russia style.

And all runs fluently with plenty of frames per second, of course. This makes me speculate what would happen if computer game programmers would develop applications for medical physics.

In the nortern part of Pripyat there is the "Prometei" cinema.

Movie theater "Prometei"

And a shop for consumer goods:

http://www.panoramio.com/photo/29016229


Housing:

http://www.panoramio.com/photo/29016249

The Duga-3 antenna complex (Nato codename "Steelyard"), is a long abandoned over the horizon radar. See e.g.:
http://englishrussia.com/index.php/2010/09/15/chernobyl-2-a-pearl-of-the-past/

Nearby the Duga steel giant they were also carrying out some ionospheric experiments, which probably inspired this part:

Note how the antenna ceramic isolators were carefully modelled.

Close to the "Duga-3" antenna complex.

And a few random screen shots from the game:

I guess those are figures from a Russian children cartoon.

"Chernobyl"

The machine to the left is a very typical soviet time water dispenser.

Inside a building in Pripyat.

The "Gastronom" welcomes you. I am afraid that pelminis are out, though.

Most Soviet housing architecture was standardized, and that way may cities tend to resemble each other a lot. I had the chance to visit Russia a few times, and always got a mild deja-vu feeling when seeing those buildings again in the game:

"Books"

Pripyat seen from below. Game is not entirely bug free...

 

Sunset at a small train depot.

Video below gives a quick tour through Pripyat city model, at varying time/weather conditions. (Never mind the split screen.)



This is the eastern part of Pripyat, it fits pretty well with google maps. (Try to enable photos, to compare some of the locations).

View Larger Map

Last but not least: a very nice trailer for the "Clear Sky" prequel (in my opinions that game still had the best atmosphere.) Video shows how to model Soviet nostalgika:



Large post, but I wanted to post this for a long time, and finally I had a Weekend off! :-)

Ripping Online Radio Streams

At work and at home I listen to online radio streams a lot (headphone of course, in order not to annoy my fellow office mates). Countless of tools exist which can save the audio stream to disk (i.e. ripping). This is perfectly legal in most countries as long as it is for private use, according to wikipedia.

A very good tool I can recommend is streamripper which can be used from command line in Linux (there are multiple windows front ends, though).
Streamripper is really "set and forget" and comes without a bloated GUI. Once you have the URL of the station you want to rip, the command line is simply:

streamripper http://www.di.fm/mp3/chillout.pls -s

The "-s" option should always be at the end. Streamripper will properly split and save the "DI Chillout" stream in individual mp3 encoded tracks. The mp3 filenames are fomatted as "Artist - Title.mp3" and to some extend the ID tags are set. The very first track will logically be stored in a folder named "incomplete" and the subsequent files will be saved at the location where you invoked the command. Very nice.

There are thousands of internet radio stations out there, these are currently my favourite ones:

http://streamer-mtc-aa04.somafm.com:80/stream/1032 - Drone Zone (Ambient)
http://www.di.fm/mp3/chillout.pls - DI Chillout
http://www.di.fm/mp3/trance.pls - DI Trance (it's crap, yet very good if office gets too noisy + coffee substitute)
http://www.di.fm/mp3/spacemusic.pls - DI SpaceMusic
http://www.dradio.de/streaming/dlf_hq_ogg.m3u - DeutschlandFunk
http://c13010-ls.i.core.cdn.streamfarm.net/dwworldlive-live/13010dwrde64.mp3 - Deutsche Welle
http://edge.live.mp3.mdn.newmedia.nacamar.net/klassikradio128/livestream.mp3 - KlassikRadio (only good in small doses)
http://www.tv-radio.com/station/france_musique_mp3/france_musique_mp3-128k.m3u - France Musique

and last, but not least:

mms://mms.online.ru/c12_1_128 - Music from 50-70ies Soviet Russia

A note about the last link: streamripper did not like the encoding or protocol of the stream - it just idles. (Right. In Soviet Russia, you don't rip stream - stream rips you.)
What works is to use (hungarian, eh?) mplayer instead:

mplayer -dumpstream -dumpfile stream.wmv mms://mms.online.ru/c12_1_128

which will save the stream into stream.wmv. But mplayer does not split the stream into individual songs, this must be done manually. I use audacity, but audacity is not always happy with the .wmv format. Mplayer can convert the wmv file to something more convenient, here I do .wav because I want to do some post-processing:

mplayer -ao pcm:file=filename.wav stream.wmv

and after importing and editing in audacity you can export the final file to mp3.

Here is an except [foo.mp3] from the very same station, which was also my motivation for going through all this. I heard this piece at a concert in the deepest Russia some years ago, and I had trouble to stop giggling while listening. Later I was told it's a very sad song about a dying swan and I was certainly not supposed to giggle, probably offending my neighbors. (I've still not identified what it is, but that should be easy to find out.)
Anyway, plenty of patriotic songs, balalaika stuff and ballades are to be found here, much of it resembles the french chanson genre. It is something refreshingly different which is to be enjoyed silently, late at night.


--- Update ---
it's Лебединая верность
http://www.youtube.com/watch?v=tgcxtyC6Qng

lyrics here:
http://www.karaoke.ru/song/6100.htm

Спасибо, Oksana. :-)

Have I been hacked?

I have this odd feeling that hacking attempts are surging in the recent months. Recently our university server was compromised and sourceforge.net was hacked twice. On my own little home server I have no valuable contents, but still it can be used by botnets for illicit activities. So I ran chkrootkit :

(...)
Checking `asp'...         not infected
Checking `bindshell'...   INFECTED (PORTS:  600)
Checking `lkm'...         chkproc: nothing detected
                          chkdirs: nothing detected
Checking `rexedcs'...     not found
Checking `sniffer'...     lo: not promisc and no packet sniffer sockets
                          eth0: not promisc and no packet sniffer sockets
(...)

Hmm.... ? UDP port 600 open?

Asking google, led me to this post...

http://www.linuxquestions.org/questions/linux-security-4/chkrootkit-checking-%60bindshell-infected-ports-600-a-653981/

This sounds very strange. My router is only forwarding ssh, http, https requests, and no UDP traffic at all, so even if a malicious demon was listening on UDP 600, outside traffic would never reach it.

What process is listening on port 600?

netstat -pane | grep 600

tells me that my portmapper rpc.statd is running and listening on that port. However, strangely enough port scanning UPD ports with nmap resulted in:

sakkub:/etc# nmap 127.0.0.1 -sU

Starting Nmap 5.00 ( http://nmap.org ) at 2011-02-05 22:40 CET
Interesting ports on localhost.localdomain (127.0.0.1):
Not shown: 994 closed ports
PORT     STATE         SERVICE
111/udp  open|filtered rpcbind
123/udp  open|filtered ntp
631/udp  open|filtered ipp
1900/udp open|filtered upnp
2049/udp open|filtered nfs
5353/udp open|filtered zeroconf

Nmap done: 1 IP address (1 host up) scanned in 1.60 seconds

What is going on? Next I copied my rpc.statd to a safe pc, and ran a md5sum on it. It was identical to the file from the uncompromised system.

I found a better explanation here:

http://www.webservertalk.com/message163183.html


which suggests it might be some orphaned port which was temporarily opened by nfs. A fix is to restart nfs:

/etc/init.d/nfs-common restart

and this eventually reassigned that port and the false positive from chkrootkit went away. So far so good.

While I was at it, I took a look at my /var/log/auth file, and was aghast on the many intrusion attempts! My poor little ssh-daemon has to cope with roughly 2000-3000 intrusion attempts per day from multiple IP numbers all over the word. Most of them try to login as root:

If you've got a root server, you're never alone. You'll have plenty of friends, from all over the world!

 
This calls for some security hardening of my sshd. For that reason I blocked root logins by setting

PermitRootLogin no
AllowUsers xxxxxx
ServerKeyBits 2048

in the /etc/ssh/sshd_config file, and restarted sshd. This link gives some more hints on how to harden sshd to the brutal world beyond the firewall:
http://www.rackaid.com/resources/how-to-harden-or-secure-ssh-for-improved-security/


In other words, I have not been hacked. But “they” are queuing in long lines, prying the door.

Rsync magic: rsync vs. scp

Accessing computers remotely is commonly made via ssh encrypted lines. In an earlier post on my (and Roy's) willworkforscience blog, I mentioned the blessings sshfs where you could mount a remote filesystem if the remote computer has an ssh daemon running.

Here I want to honor a very old program which is called "rsync" which can be used for fast copying of files to/from remote computers. Rsync has been extended with plenty of new functionalities, and generally I prefer rsync to scp when I have to copy more than one file. Let me give a few examples:

Example 1): speed

In my last post I presented a little script which generates an arbitrary amount of files with random contents. For this test I generated 1000 files which were 1k large each. Copying them from my home PC kepler to the remote computer lululu (via a 10/2Mbit ADSL line) will be signifcantly slower with scp than rsync:

bassler@kepler:~/test$ time scp * lululu:test/.
bassler@lululu's password:

real   1m19.619s
user   0m0.188s
sys   0m0.204s

bassler@kepler:~/test$ time rsync * lululu:test/.
bassler@lululu's password: 

real    0m14.780s
user    0m0.048s
sys    0m0.032s

That is 80 seconds for scp and 15 seconds for rsync. (Yes, the target directory was wiped empty before I issued the rsync command.) The overhead of scp becomes significant if many files are to be copied. If I copy a single 1M file instead of 1000 1k files, then there is no noticeable speed difference.

Example 2: transferring very large files

Rsync has a nice resume option for transferring large files, if transfer for some reason is interrupted. Basically what your options should look like:

rsync -rvv --inplace --append --progress  /disk1/Movies/* lululu:Movies/.

afaik, this is not possible with scp in a straightforward way.

Example 3: tunnel your way past firewalls and gateways

This is probably the coolest feature of rsync! If I am at home and want to update the files on my office PC baslup I face the problem that the PC is hidden behind the university firewall/gateway lululu. So with scp I would have to copy the files to lululu first, and from there again copy it to my office PC baslup.
Now, if rsync is installed on all PCs in the chain, I can issue a very simple command which will establish a link directly to my office PC.

rsync -va -e "ssh lululu ssh" test/* baslup:test/

You can add several PCs in the chain. Here fufufu is added:

rsync -va -e "ssh lululu ssh fufufu ssh" test/* baslup:test/

the only annoying thing is, you have to enter your password multiple times, (unless you add the ssh keys for password less logins).

Ok, finally a list of the most important options for rsync:

 -v verbose
 -r recurse into dirs (you might just as well use -a instead)
 -a archive: recurse into dirs, preserves symlinks, permissions, timestamps, group and owner.
 -u update: don't overwrite newer files on receiver
 -n dry run, copies no files, but shows what would be done
 --del delete files/dirs from remote server which do not exist locally
  -z enable compression. Yes, if you have files with lots of air in them, you can achieve dramatic transfer speedups. Works well for uncompressed scientific data, bitmap files, large textfiles etc, but is much slower when transferring mp3, jpg, gz, zip, pdf and similar compressed files...


Thank you, developers of rsync! This has saved me alot of hassle.

Syntax highlighting of code in blogger

I have not been too happy with the lack of code highlighting in blogger. For example, here is a little script I wrote "mkfile.sh" which creates a bunch of files with random contents. Switching to the HTML editor and encapsulating the code in <pre> </pre> html tags results in:

#!/bin/bash

if [ $# != 3 ]
then
  echo "Create a bunch of files with random contents."
  echo "Usage:"
  echo "$0 filename number size"
  echo "  filename: basename of the file"
  echo "  number  : number of files to be created"
  echo "  size    : file size in bytes"
  exit   
fi

for (( i = 0 ; i < $2 ; i++ ))
do
  echo "Writing $3 bytes to $1.$i"
  dd if=/dev/urandom of=$1.$i bs=$3 count=1 > /dev/null
done

which is not all that readable. There are code viewers on the web which converts the code to nicely formatted HTML, a quick googeling directs to e.g. this one:
http://codeformatter.blogspot.com/2009/06/about-code-formatter.html

 #!/bin/bash  
 if [ $# != 3 ]  
 then  
  echo "Create a bunch of files with random contents."  
  echo "Usage:"  
  echo "$0 filename number size"  
  echo " filename: basename of the file"  
  echo " number : number of files to be created"  
  echo " size  : file size in bytes"  
  exit    
 fi  
 for (( i = 0 ; i < $2 ; i++ ))  
 do  
  echo "Writing $3 bytes to $1.$i"  
  dd if=/dev/urandom of=$1.$i bs=$3 count=1   
 done  

Right, formatting has improved, but there is still no syntax highlighting. The best solution I could find is the one at http://tohtml.com

#!/bin/bash

if [ $# != 3 ]
then
  echo "Create a bunch of files with random contents."
  echo "Usage:"
  echo "$0 filename number size"
  echo "  filename: basename of the file"
  echo "  number  : number of files to be created"
  echo "  size    : file size in bytes"
  exit   
fi

for (( i = 0 ; i < $2 ; i++ ))
do
  echo "Writing $3 bytes to $1.$i"
  dd if=/dev/urandom of=$1.$i bs=$3 count=1 
done

which looks the way I wanted it!

Now I just have to figure out what to do with thousands of 1k garbage files flocking around in my directory.

A few notes on Andrei Tarkovsky

One of the most famous russian film directors is inevitably Andrei Tarkovsky. During his career he only shot 7 films, most noteworthy the autobiographic movie Зеркало "The Mirror" and the two science fiction movies Солярис "Solaris" and Сталкер "Stalker".

Tarkovsky tries to emphasize the cinema genre as an art form. Generally you can consider his movies as paintings with a time dimension added to it. For instance in "The Mirror" Tarkovsky let a famous paintings by the flasmish painter Bruegel come alive. "The Mirror" is a complicated film though, and personally I like "Stalker" the most with its strong meditative and post-apocalyptic feeling. This is not only expressed by the photography and the story line, but also by the sound design: in the scene shown below, the three main characters enter the zone on a motorized draisine:



Note, how the engine sound fades away, and only the sound of the rail joints remains, slightly electronically modified. Epic: their wrinkled faces, and a very characteristic Solonitsyn dozing off. The black and white shooting adds contrast emphasizing every little fold and the battered souls. When they finally enter the zone, the movie switches from B/W to colour:



"Urban Exploration" is about visiting recent ruins of our civilization, and that concept has surged during the last 5 years, becoming widely popular among people on my age. Arranged trips can be bought places such as Chernobyl. I prefer the term “Industrial Wasteland Tourism".

Tarkovsky truly was ahead of his time with the post-apocalyptic scenography in Stalker. Along with the absence of special effects which tend to look cheap after some time, this makes the footage timeless.

Our most noteworthy contemporary danish director Lars von Trier ("Antichrist", "Dogville") is heavily inspired by Tarkovsky (also his "Europa" trilogy). Checkout this scene from "Antichrist" :



the scenography, even the colour balance is very Tarkovsky-ish, there is a similar scene in "The Mirror", but I could not find it on youtube. :-/

Last year there was a one-day course dedicated to my favourite director, arranged by FOF in Aarhus. The program involved a lecture on Tarkovskys life and creative portfolio and watching the movie "The Mirror" followed by a discussion. "The Mirror" is probably the movie which is most difficult to understand. It is full of biblical references and encrypted symbolism, I have real trouble to uncover all layers. However, this is of course challenging and makes the movie even more interesting. So, this course seemed very promising, and I signed up. Did some preparations and tried to group all the questions I wanted to discuss.

Of course, if you have high expectations, chances are high you get disappointed. Apart of me, 6 people attended. The lecturer was a teacher of a Russian language course, and all participants (except me) where students of her class. Unfortunately, I was the only one who had seen more than just one movie (and more than just once...). The discussion part was therefore rather halting.

Yet, what I found very interesting was what the teacher said about how the films were received in Soviet Russia at that time. She was a teenager during the 70ies, and generally what was shown in movie theaters was daft simple minded movies. (I.e. imagine all you were allowed to see was mainstream hollywood movies! Gaaaahh...!) The Tarkovsky movies tended to be very controversial, they were not banned though. Instead they where rarely shown, but if they were, people flocked to the movie theatres. She and her friends brought paper and pencil into the cinema and took notes in the dark of the poems occuring in Tarkovskys movies.

As mentioned earlier, Tarkovsky tried to emphasize the art element in cinematography. The pool scene (yeah, we had it in an earlier post, music is by Artemiev) then must represent the classic still life discipline in painting!




All Tarkovskys movies can be watched here for free:

http://www.openculture.com/2010/07/tarkovksy.html

Later I plan to add a post about the STALKER computer game series made by Ukranian GSC. They are based on the very same movie, and certainly worth a closer investigation.