Reading temperature with WR703N and DS18B20 on OpenWrt

 
You can easily get temperature readings from external Dallas DS18S20 sensors (1-wire protocol) on your TP-Link WR703N.
 
First things first, grab your credit card and order few Dallas DS18S20 sensors and usb to serial adapters. Now you need to connect Dallas DS18S20 sensor to USB to serial adapter.
 
After you got hardware part sorted now you need to setup software part. Main tool for reading temperature from 1-wire sensors is digitemp. Digitemp uses bit-banging to emulate 1-wire protocol over serial port.
 
opkg install digitemp
 
You also need usb to serial kernel module (driver) so install that also:
 
opkg install kmod-usb-serial-cp210x
 
After digitemp is installed you need to search 1-wire bus for all connected sensors, each sensor has it’s unique address so is has to be discovered and added to config file with this command:
 
digitemp_DS9097 -s /dev/ttyUSB0 -i
 
Now you can read temperature from your sensor:
 
digitemp_DS9097 -a -q
 

How SpiderOak deleted my files!

 
I have been happy user of SpiderOak until now, this happened today. I have two machines (let’s call them Ana and Bill) that sync one folder, and on first one (Ana) I added one new folder with arround 2GB of data to upload, as it was over 512Kb/s upload link I left it over night.
 
Next morning I saw that first Ana has finished upload, but I couldn’t find data in synced folder on Bill, that was strange, that neven happened before. When I noticed that Bill was acting strangly (Firefox freezing up) I investigated and saw that I ran out of space on my /home partition. No big deal, I moved one directory to other disk and restarted SpiderOak.
 
I also noticed one really big file called something like “SpiderOak cache” or something similar. From that it was obvious that SpiderOak started syncing the directory, but but it looks like SpiderOak doesn’t sync one file at a time but whole directory. Sync process got stuck when it ran out of space.
 
When I restarted SpiderOak that huge file on Bill was gone, but then then I looked on Ana and the original directory has also been removed, now alarms started doing off! It looks like sync went in wrong direction. Instead of transfering data from Ana to Bill, when SpiredOak got stuck on Bill and didn’t transfer whole directory then deleted temporary file but also sent command to delete that directory back to Ana!
 
This is really bad!
 
I’m reading on SpiderOak blog that data is never deleted, but I can’t find it anywhere! Help! Where is this “Garbage bin” that they talk about?
 

Loši dani za twitter klijente pod Linuxom

Twitter je umirovio v1 API što je većinu Twitter klijenata pod Linuxom učinilo neupotrebljivim. Do danas sam koristio Hotot, tolerirajući mu bubice i stvari koje nikada nisu radile kako treba (filteri). U repozitoriju nema traga o prilagođavanju novom apiju.

Gwibber također ne radi. Choqok ne radi, ali uskoro bi ga trebali srediti. Turpial koji se nalazi po repozitorijima također ne radi, ali ispravak bi trebao uslijediti.

Polly radi, ali ne verzija koju je moguće instalirati iz distribucija već treba instalirati unstable ili daily iz ppa repozitorija. Polly mi je ružnjikav, za neke osnovne operacije uvijek je potreban klik više.

Birdie najviše podsjeća na Hotot, ali neodostaje mu mogućnost prilagođavanja, ne može prikazati diskusiju.

Nekad popularni Twitter klijenti koji su koristili Adobe AIR se više ne koriste jer isti više nije podržan pod Linuxom.

Nakon brze štih probe spomenutih klijenata otkrio sam da Hotot, koji se instalira kao Chrome aplikacija, ipak radi, pa sam se ipak odlučio na kompromis. Razlike između pravog Hotota i ovog chromiranog skoro da i nema jer je i u jednom i u drugom slučaju kod isti (JavaScript, HTML, CSS) samo je omotač drugi.

I takon malo testiranja ostaje status quo. I zato…

Apple će uskoro prefarbati jabuku

Nakon što su ušarenili novi iOS, izgemištirali ga tražeći motivaciju od svih konkurenata po malo, jedino što im je ostalo da ponovno vrate boje u logo. To će valjda napraviti sljedeći put, ovaj put bi to bilo previše. Šarenilo broja 7 to daje naslutiti. Kad sam da vidio prvo što sam pomislio kako to izgleda kao odbačeni dizajn s promocije Windowsa 7.

Izgleda da se Apple magija razilazi kao magla. iOS 7 in action više nalikuje nekoj wannabe Windows Phone 8 temi za Android koja se inspirirala WebOS-om. Sad mi više ni copycat teme na GNOME-Look ne izgledaju tako loše.

Tagiranje datoteka

Novi OS X Mavericks će omogućiti tagiranje datoteka u Finderu. Nautilus to već ima možda cijelo desetljeće. Nije baš neki killer feature kojeg će sad korisnici početi masovno koristiti. Znam to, probao sam. Sve će stati na nekolicini pedantnih gnjavatora. Da je Jobs živ ovo tagiranje bi drugačije izgledalo. Ne bi korisnici trebali sami tagirati datoteke, OS X bi to radio umjesto njih. Većina datoteka u sebi sadrži dovoljno meta podataka na osnovi kojih bi se mogle označavati, kategorizirati. Prepoznavanje članova obitelji na slici danas nije SF feature.

iOS i OS X korisnici ulaze u novo doba. Jednostavnost se gubi, složenost se vraća, a Tim Cook mjerka kantice s bojom.

Jacks and Masters

I haven't written a single word this year. It's a busy one for me building, scaling and supporting more big sites. Interesting problems were solved, bugs were found... but still, I didn't feel like I stumbled onto anything worthy of publishing that hasn't been rehashed a 1000 times already through blogs and eventually change-logs. But thinking about my lack of material, while catching up on my podcast backlog gave me an idea to write something about the sysadmin role in all this.

Many times in the last year I returned to two books as best practices guides for building and scaling web-apps. Those are Scalability Rules and Web Operations. I recommend these books to any sysadmin interested in web operations, as they share experiences from engineers working on the top sites and there's just no other way to gain these insights unless you join them.

That brings me back to my podcast backlog. Episode 38 of the DevOps Cafe had a very interesting guest (Dave Zwieback) that talked a lot about hiring sysadmins, and generalist vs. specialist roles in systems administration today. I am a generalist which in it self is fine, but there's a big difference between "jack of all trades, master of some", and being a "master of none". I've been thinking about it since the podcast, as I wasn't thrilled with what I was doing lately, that is jumping through a lot of new technologies to facilitate all kinds of new frameworks web developers use. Often times that means skipping some kind of "natural" learning course, instead you learn enough to deploy and manage it, while the real knowledge comes within a certain period of time spent debugging problems when it breaks apart.

Now to tie both parts of the text together. If you want to join site reliability engineers at one of the top sites how do you justify drawing a blank when asked to explain how Varnish malloc storage works internally, if claiming you built huge caches with it? The Jack issue is amplified if you consider there is now a first, or even a second, generation of sysadmins who have never stepped into a data-center and are missing out on hardware and networking experience. Appropriate name that comes to mind is "the cloud generation", and I'm a part of it.

USB IR Toy - PIC 18F2550 fw_update under Linux

I'm in the process of setting up my HTPC, so I ordered USB IR Toy to be able to turn TV on without using remote control.

When I got it, I was unable to make it work. Fortunately, there is V command which returns current version of firmware, and on my device it was V212 which means Hardware V2 and firmware V12. Since latest firmware is V22, I decided to upgrade firmware. In the process I bricked it (using wrong fw_update program which doesn't flash whole device, but just first 2K of application). Fortunately, this PIC has Diolan bootloader which is accessible using jumper (small blue peace of wire on picture) even if you flashed just part of application on it (as I did).

Result it fw_update for PIC 18F2550 which IR Toy uses , updated wiki page about IR Toy firmware update on Linux and comment on bug tracker. Hopefully, this will help someone else instead of learning all this stuff, which was fun in itself :-)

Next step was to try SUMP logic analyzer mode which can produce nice graph of IR communication. I used Debian sump-logicanalyzer package to produce following picture:

Now that I was sure that it works and can communicate with my LG TV remote, I decided to give one of sample programs, IRToyRecPlay a try. It doesn't come with Makefile, so I compiled it using:

gcc *.c -o IRToy

Unfortunatly, it doesn't seem to work for me (can you see pattern here?). So, I decided to try fork of IRToyRecPlay on github which works better under Linux (after PIC fw_update problems I didn't really spent any time investigating why - I suspect that upstream Dangerous Prototypes source works well under Windows).

First, I recorded power button from remote using

./IRToy -d /dev/ttyACM0 -f lgtv -r

This created lgtv_000.bin file which I could reply to get TV turn on:

./IRToy -d /dev/ttyACM0 -f lgtv -p

I noticed that I can't just reply same sequence again to turn TV off, so more works needs to be done... But that's only software problem, so I did my first clash with PIC hardware somewhat well :-)

Update: I had problem with play of IR codes which would die after few tries. It seems that firmware V2x doesn't support USB suspend correctly, so easiest solution is to use powertop to turn off auto-suspend for whole usb bus (including host controller!) on which IR Toy is connected. This will change a bunch of values in grep . /sys/bus/usb/devices/*/power/control from auto to on. But you will have to re-plug you IR Toy after that so better solution is echo "options usbcore autosuspend=-1" > /etc/modprobe.d/irtoy.conf and this will turn autosuspend for all usb devices.

X11 running on Nook Color without Android stack

I have been toying around with idea of having real Linux stack (X11 and friends) on Nook Color. While this seems like a silly thing to do, it does allow me to use x2x and transfer keyboard and mouse from my laptop to tablet which is handy. If also allows me to run X11 applications on tablet screen using DISPLAY=nook.lan:0. I was fortunate enough to find blog post how to run Linux framebuffer X server on Android but I wanted to use touchscreen so I decided to write my own xorg.conf (this brings back memories...).

To get full-blown Debian-based distribution on your Android take a look at BotBrew Basil. It's Emdebian based distribution which will setup mount points and various other stuff so you don't have to do that manually. Since it's Debian based, you are not limited to Emdebian packages -- you can (and will have to) add normal sid:

(BotBrew)root@localhost:/# cat /etc/apt/sources.list.d/sid.list 
deb [arch=armel] http://ftp.debian.org/debian sid main contrib non-free

If you want to know more about Emdebian hop over to DebConf 12: Integrating Emdebian into Debian [video].

With all this prepared, we are ready to shut down Android stack:

adb shell setprop ctl.stop media
adb shell setprop ctl.stop zygote
adb shell setprop ctl.stop surfaceflinger
adb shell setprop ctl.stop drm

Next step is installation of required packages:

dpavlin@t61p:~$ adb shell
root@android:/ # TERM=xterm chroot /data/botbrew-basil/ /bin/bash --login
(BotBrew)root@localhost:/# apt-get install xserver-xorg-video-fbdev xserver-xorg-input-evdev \
   xserver-xorg-input-multitouch x11-xserver-utils xinit \
   matchbox matchbox-keyboard xterm

I decided to use matchbox, mostly becuase it's only window manager which comes with on-screen keyboard which is useful on touch screen device.

After installation you will need to setup X symlink and create .xinitrc:

root@android:/ # ln -s /usr/bin/Xorg /usr/bin/X

root@android:/ # cat ~/.xinitrc                                              
( sleep 1 ; matchbox-keyboard -o portrait ) &
xhost 192.168.1.61
matchbox-session

Finally, you need to create xorg.conf:

Section "ServerLayout"
    Identifier    "Layout0"
    Screen        "Screen0"
    InputDevice   "cyttsp-i2c" "CorePointer"
    InputDevice   "gpio-keys" "CoreKeyboard"
    InputDevice   "twl4030-keypad" "CoreKeyboard"
EndSection

Section "InputDevice"
    Identifier    "gpio-keys"
    Driver        "evdev"
    Option        "Device" "/dev/input/event0"
    # code 102 (KEY_HOME)
    # code 116 (KEY_POWER)
EndSection

Section "InputDevice"
    Identifier     "twl4030-keypad"
    Driver         "evdev"
    Option         "Device" "/dev/input/event1"
    # code 114 (KEY_VOLUMEDOWN)
    # code 115 (KEY_VOLUMEUP)
EndSection

Section "InputDevice"
    Identifier     "cyttsp-i2c"
    Driver         "multitouch"
    Option         "Device" "/dev/input/event2"
    # mouse should move as fast as finger and not faster
    Option         "AccelerationScheme" "none"
    # evdev has it, multitouch doesn't so it behaves like touchpad
#   Option         "IgnoreRelativeAxes" "True"
EndSection

Section "Device"
    Identifier    "Card0"
    Driver        "fbdev"
    Option        "fbdev" "/dev/graphics/fb0"
    # rotate screen to be in sync with touchpad orientation
    Option        "Rotate" "CCW" # CW=90 UD=180 CCW=270
EndSection

Section "Screen"
    Identifier    "Screen0"
    Device        "Card0"
EndSection

This will map all hardware keys and use mutitouch driver for screen. To make it work, I used evtest package which allows you to see events from input devices so you will know which device produce keyboard events and which produce multitouch events. To be honest, this solution isn't prefect, because screen behaves like touchpad, so you can't just point to screen and expect your cursor to just to that position.

Following video shows X server in action.

This is simple unaccelerated frame buffer. This makes performance less then desirable. There are a few implementations of OMAP xorg server:

• xf86-video-omapfb uses DSS kernel support which seems to be part of CM kernel, so this might be good next thing to try out
• xf86-video-omap is newer implementation, but this requires 3.3 kernel and is not yet stable.

Having accelerated OMAP X server and fixed touchscreen issues would make Nook somewhat nice Linux tablet, if only it isn't so heavy for day-to-day use :-)

DORS/CLUC 2013 conference

Last week we had another annual conference about Free Software and Open Source DORS/CLUC 2013. For a last year, I was playing with various hardware, so this year, I was part of Internet of things panel (talking about ARM based machines), and I had lecture about GNU/Linux on ARM devices for $50-$100.

I also submitted hardware workshop for program which got accepted, so I quickly realized that I'm really no hardware expert and that I could use some help to make interesting workshop. Fortunately I have a few good friends who know more about hardware that I ever will, so I summoned Lovro and Dalibor to help me cover hardware and antenna design. Few weeks ago I was at NSND Belgrade 2013 where I had good fortune to meet Filip who is working for Dangerous Prototypes. I couldn't really believe my good luck since I wanted to talk about Bus Pirate a great multi-purpose tool which got me into hardware in the first place. So, in the end, I had three very skillful people to back me up in this workshop which was hopefully useful and interesting to people attending it. For future reference, I will include a few links below about topics we convered:

• Filip started with Bus Pirate including it's supported protocols I2C, SPI, 1-wire, UART, JTAG (and others) and example of using it to communicate with I2C EEPROM
• Dalibor make great presentation about antenna design, including practical examples for ADS-B and 443.92 antenna (simple one from coax cable and one with RF filter for PiHAT) while I tried to explain why RTL-SDR DVB-T sticks made it possible to listen to various your IMS sensors using rtl_443.
• Next, Filip again took stage to speak about Open Hardware and possibilities to manufacture and sell such devices based on expirience of Dangerous Prototypes cooperation with Seeed Studio.
• In last part of our workshop we played with blinkm, took questions and discussed other Dangerous Prototypes projects which we had with us which included USB Infrared Toy, Bus Blaster and Open Bench Logic Sniffer but we also talked and showed Arduino and Mojo FPGA board which we had us also to ZPUino which is interesting combination of Papilio FPGA and Arduino IDE. Lovro did a nice overview of FPGAs and how are they different from software programming.

As you can see from notes above, workshop was a mix of different projects but hopefully it managed to convey my excitement about current moment in time where you can hack hardware even without taking soldering iron (and burning your fingers). If you do take soldering iron, please make your project Open Source Hardware...

Prognoza

Najjednostavniji način za prognoziranje budućnosti je da ju stvaraš. Ali malo tko je u takvoj prilici. U IT svijetu prognoze su obično nečije želje, nevješte igre brojkama, pokušaj utjecaja na tržište ili smiješni pokušaji pojedinih evanđelista da pogode nešto što ponekad ni sami dovoljno ne kuže.

U koju grupu pripada Thorsten Heins, Blackberry CEO? On tvrdi da za pet godina neće biti razloga za posjedovanje tableta. On je u grupi onih koji su mogli utjecati na budućnost, ali nisu to uspjeli i sada traži opravdanje. Mogli bi sa sigurnošću prognozirati da za 5 godina Thorsten Heins neće biti CEO. Ili će se proizvodnja pametnih telefona s tipkovnicom pokazati kao pravi pogodak u jednu usku tržišnu nišu. Prodaja Q10 uređaja je dobro krenula. Toni Ahonen negdje već piše post o tome kao velikoj prilici koju je Nokia propustila. Tek je na pola, još mu fali pedesetak kartica, bude ga objavio sutra.

Compiling tunneldigger client on Fedora

 
Compiling tunneldigger client on Debian/Ubuntu is pretty straight forward if you follow official instructions.
 
For Fedora I just needed to find appropriate package names and install them first:
 
sudo yum install iproute bridge-utils libnetfilter_conntrack python-devel libevent-devel ebtables libnl-devel python-pip
 
then you just need to pull code from git:
 
git clone git://github.com/wlanslovenija/tunneldigger.git
 
and then just compile tunnel-digger client:
 
cd tunneldigger/client
make
 

Tweak Wireless Power Output on OpenWrt and Linux

 
This post is just to remind me later of few misc openwrt wireless commands.
 
First there is no iwconfig you need to install it:
opkg update
opkg install wireless-tools
 
iwconfig command shows current wireless power outpu:
# iwconfig
wlan0 IEEE 802.11bgn Mode:Master Tx-Power=20 dBm
 
Maximal power is defined by multiple capability of your hardware and by laws in your country (regulatory domain).
 

Check your current regulatory domain via / etc/config/wireless file or just use this command:
# iw reg get
country US:
(2402 - 2472 @ 40), (3, 27)
(5170 - 5250 @ 40), (3, 17)
(5250 - 5330 @ 40), (3, 20), DFS
(5490 - 5600 @ 40), (3, 20), DFS
(5650 - 5710 @ 40), (3, 20), DFS
(5735 - 5835 @ 40), (3, 30)

 
I don’t live in USA so I’ll change this to my country:
# iw reg set HR

# iw reg get
country HR:
(2402 - 2482 @ 40), (N/A, 20)
(5170 - 5250 @ 40), (N/A, 20)
(5250 - 5330 @ 40), (N/A, 20), DFS
(5490 - 5710 @ 40), (N/A, 27), DFS

 
You need to be careful because some countries like BO allow setting upto 1000 mW (30 dBm)! But still if your hardware is not capable of such high power that you can’t push it over the limit.
 
Check wireless capability of your hardware via iw command:
iw phy0 info
 
Manualy set output power to 50mW (17dBm):
iw phy phy0 set txpower fixed 17000
or
iwconfig wlan0 txpower 17
 

Google Keep? Ignorirati!

Google je pokreno novi servis. Keep. Par dana prije toga je najavio gašenje Readera. Kako mogu očekivati podršku korisnika, a pokazali su da im do njihovog mišljenja nije stalo?! Prošle godine je ugašen Notebook. Na neki način Keep je zapravo rebootani Notebook. Koliko vidim Keep je jednostavna aplikacija. Ali ne onako Apple jednostavna već Google jednostavna. Kao da je netko to napravio u onih 20% ili koliko već legenda kaže da Google djelatnici mogu raditi što hoće. Google može Keep gurati koristeći silu. Kao što to radi s Plusom. Ja samo znam da ga neću koristiti. Za bilješke i ideje sam počeo koristiti Trello. Veća je vjerojatnost da ću s njima i nešto napraviti. U Keepu bi se one utopile, kao što je bio slučaj i s Notebookom i ostalim sličnim servisima koje sam koristio.

Google Reader - kraj za novi početak?!

Google Reader koristim tako dugo da sam već i zaboravio da li sam koristio nešto drugo. Desktop RSS readeri su mi vrlo brzo postali nepraktični jer sam koristio nekoliko računala i bilo mi je vrlo važno da reader “zna” da li je neki članak već pročitan. Zapravo jako rijetko čitam ono što dolazi u feedu, pročitam naslov, prvih par redaka, skeniram tekst i tek onda odlučujem hoću li pročitati tekst. Ponekad je to možda svaki deseti tekst, a kad se feed napuni onda preskačem i veći broj. Odlična stvar kod Google Readera su mi tipkovničke kratice koje mi omogućavaju brzu navigaciju prilagođenu mojem načinu čitanja, tj. skeniranja. Kad odlučim da je tekst vrijedan čitanja obavezno odlazim na originalni url.

Ima par iritirajućih stvari kod njega, a imam i par ideja koje bi učinile učinkovitim moj način čitanja. Pokušao sam mu pronaći alternativu, ali bezuspješno. Najavljeno slanje Google Readera na groblje ipak me primorava na odabir. Zamjene iskaču iza svakog ćoška, svi portali koji imaju tech rubriku nude rješenja, ali jedna od glavnih zamki se krije u činjenici da dobar dio njih koristi Google Reader kao backend (npr. feedly).

Najviše me privlači neko rješenje koje bi mogao instalirati na vlastiti poslužitelj pa da ne moram strahovati od prisilnih ugrobarenja. Kao takav se reklamira Fever, cijena je pristupačna (30$), ali činjenica da nemaju demo (’a live demo isn’t possible’, wtf?) malo me odbija. Video nudi malo više informacija, ali nedovoljno da za prihvaćanje rizika.

U raznim preporukama često se spominje i NewsBlur. Zbog velikog interesa trenutno se moguće registrirati samo za premium pristup. Već sam ga i prije škicao, ali djelovao mi je nekako prešareno s previše zahtjeva za klikanjem. Ono što većina ne spominje kad preporučuje NewsBlur je činjenica da je open source i da mu je izvorni kod dostupan na GitHubu. Meni je još zanimljiviji jer je riječ o Django projektu. Ako se dosad još niste sreli s pojmovima Django, Celery, RabbitMQ, MongoDB instalacija bi vam mogla predstavljati veliki problem.

Za brzo skeniranje/konzumiranje sadržaja zadnjih par mjeseci sam koristio Zite. To nije klasični RSS reader, ali zadovoljava glad za informacijama. Nema verziju za desktop niti za Android tablete i to mu je veliki nedostatak. Neka mješavina Zite sučelja i RSS readera s bazom i stalnim osvježavanjem i analiziranjem feedova je ono na što bih želio prijeći kad se Google Reader ugasi. Postoji li nešto takvo?

http://abstrusegoose.com/503

Gov.hr - ušteda ili bacanje novca u vjetar?

Netokracija je donijela vijest o tome kako će novi Vladin portal olakšati pristup informacijama i uštedjeti dva milijuna kuna. Prva činjenica koja bode oči je da se za portal Moja uprava trenutno izdvaja 2,8 milijuna kuna godišnje. To je ogroman novac. Nema sumnje da na tom portalu ima korisnih informacija, ali nije mi jasno koje je opravdanje za tako velike troškove. Oni bi mogli imati 10 ljudi s plaćom od 20.000 kn bruto i još bi ostalo za najbolji hosting i ostale troškove. Takva ekipa bi mogla napraviti odličan portal na kojem bi se informacije nadopunjavale iz minute u minutu. Očito je da to nije slučaj. Ali možda ipak portal koristi jako puno ljudi jer im je jako koristan? Pokušao sam uz pomoć Alexe i gemiusAudience podatka procijeniti broj jedinstvenih posjetitelja uspoređujući stranice koje su slično rangirane. Zaključio sam da portal Moja uprava, koji godišnje troši 2,8 milijuna kuna, ne posjećuje više od 2000 jedinstvenih posjetitelja mjesečno. To je strašno rasipanje novca za tako mali učinak.

Prijedlog: Sve web stranice javnih servisa moraju objavljivati troškove

Besmislenom trošenju treba stati na kraj. Svi se kunu u tu nekakvu transparentnost, a skrivaju podatke kao zmija noge. Predlažem da se za ovakve web stranice uvede obaveza da na samoj stranici objavljuju podaci (i redovno ažuriraju) tko je uključen u projekt, što konkretno radi na projektu, koliki su njegovi troškovi (plaća, honorar) te tehničke i financijske informacije o aplikaciji i hostingu. Statistika posjećenosti stranice bi trebala biti svima javno dostupna kao i podaci o učinku.

Ako se poduzetnici mogu kažnjavati zbog apsurdnih zakona onda se javna uprava mora natjerati da opravda utrošeni novac te da radi bolje, pametnije i jeftinije uz povećanje učinka.

Dobar, loš, EOJN

@VladaRH na Twitteru papagajski zove ljude na natječaj za gov.hr. Primjenjujući najbolju praksu leopardove šape učinili su dokumentaciju teško dostupnom. Da bi joj pristupili morate biti prijavljeni. Pokušajte napraviti korisnički račun. Da li vam je uspjelo? Meni nije. Nakon unesene adrese kliknuo bih na gumb Dalje, ali ništa se nije događalo. Pokušao sam iskoristiti formu za zaboravljenu zaporku pretpostavljajući da je korisnički račun možda napravljen ili sam već prije napravio korisnički račun (često bi se događa da se pokušam prijaviti na neki servis i saznam da sam se registrirao prije par godina), a nisam to nigdje zapisao. Umjesto klasične forme za te slučajeve, dočekalo me ovo. Ako takav jednostavan slučaj nije odrađen na jednostavan način siguran sam da je svakodnevno korištenje aplikacije za naručitelje isti takav usability horor.

Svojevremeno se izvođač EOJN-a hvalio na jednoj konferenciji ovim ostvarenjem, a ako se dobro sjećam naziv prezentacije je bio u stilu “Možemo bolje i od Rakara”. To je kao trebao biti bolji odgovor na Provjetravanje javne nabave. Ali pokazalo se da su napravili lošiju aplikaciju od one stare iz koje je Rakarov tim preuzimao i kopirao podatke.

Prijedlog: Svi javni servisi na javni usability test

Web aplikacije javnih e-servisa moraju biti jednostavne i intuitive na korištenje. Postojeći primjeri pokazuju da to nije slučaj. Trebalo bi uvesti obavezu da svi javni e-servisi moraju proći javni usability test, tj. da svi zainteresirani mogu testirati aplikaciju (ili klikabilni wireframe) i dati povratnu informaciju. Ti projekti su preskupi i prevažni da bi samo tako objavljuju bez ikakvog testiranja i mogućnosti da se greške poprave.

Prijedlog: pojeftiniti javnu nabavu

Pogledajte cijene objave u Elektroničkom oglasniku Javne nabave Republike Hrvatske. Naručitelj za jedan postupak treba izdvojiti najmanje 1520 kn (obavezni su mu poziv na nadmetanje i obavijest o sklopljenim ugovorima). Pazite, to je cijena samo za objavu u Elektroničkom oglasniku. Objavu teksta u tiskanom izdanju Narodnih novina naručitelj plaća posebno po standardnim cijenama. Kakvi su to troškovi da naručitelj za jednu objavu mora izdvojiti tolika sredstva?

Ne treba se onda iščuđavati zbog velikog broja bagatelnih nabava (do 70.000 kn) pojedinih naručitelja koji usitne nabavu kako bi izbjegli postupak javne nabave i obvezu njezine objave. Neki naručitelj izbjegavaju objave zbog koruptivnih aktivnosti, ali neki ju izbjegavaju zbog štednje. Grad ili općina koji 100 postupaka male nabave pretvaraju u bagatelne nabave godišnje štede 152000 kn. Cijenu objave u EOJN-u treba smanjiti na razuman iznos kako izbjegavanje postupka ne bi bilo potrebno.

Elektronički oglasnik javne nabave je u ovom trenutku preskup, monopolistički servis koji ne ispunjava svoju osnovnu svrhu, a ona je osiguravanje ekonomičnih i učinkovitih postupaka javne nabave te racionalno trošenje novca poreznih obveznika. Sprječavanje korupcije i kriminala ne može biti jer nedostaju neki osnovni elementi koje je Rakar pokušao prikazati svojim provjetravanjem.

Formatting output from Raspberry Pi temperature sensors

 
Raspberry Pi crew released tools and drivers than enabled reading from CPU and GPU temperature sensors.
 
In order to read GPU temperature use:
$ /opt/vc/bin/vcgencmd measure_temp
$ temp=43.3′C
 
Reading CPU temperature is similar, just read value from sys file system:
$ cat /sys/class/thermal/thermal_zone0/temp
$ 43850
 
As you can see both outputs need additional formatting to get only number values from them, and if you wish to have decimal precision then it gets a bit more complicated…
 

GPU Temperature
Formatting GPU temperature output can be done with cut command, so lets show only characters from 6 to 9.
 
$ /opt/vc/bin/vcgencmd measure_temp|cut -c6-9
$ 43.3
 
CPU temperature
Acquiring and formatting CPU temperature is similar procedure but with a twist…
 
$ cat /sys/class/thermal/thermal_zone0/temp
$ 43850
 
CPU temperature is shown without decimal point, but it is obvious that numbers after first two are decimal.
 
If you just divide by 1000 in bash you loose precision of decimal places because bash doesn’t work with decimal numbers :(
 
$ echo $((`cat /sys/class/thermal/thermal_zone0/temp`/1000))
$ 43
 
So lets try something else…
$ echo $((`cat /sys/class/thermal/thermal_zone0/temp|cut -c1-2`)).$((`cat /sys/class/thermal/thermal_zone0/temp|cut -c3-5`))
$ 43.312
 
Or you can use variable to get same effect:
CPUOUT=`cat /sys/class/thermal/thermal_zone0/temp`
CPU=`cut -c1-2 $CPUOUT`.`cut -c3-5 $CPUOUT`
$ 43.312
 
This prints first two digits, then decimal point and lastly last three digits. If anybody has other solution please share it in the comments...
 

NSND 2013 Osijek : RTL-SDR software defined radio

For a long time, doing anything with various devices which use radio waves proved to be almost impossible for software geeks like me. Fortunately with RTL DVB-T sticks all this is about to change, and at our last meeting Ništa se neće dogoditi in Osijek I tried to show some of possibilities using them.

For a start you need compatible Realtek RTL2832U DVB-T tuner. Just go over to rtl-sdr osmocomSDR and examine supported tuner list. Choice of DVB-T sticks in Croatia isn't so good. They are about 170 kn and come with FC001[23] tuners: Hama 53176 has FC0013 tuner and TV LV5TDLX has FC0012 tuner. I did some research and ordered this $13 DVB-T from dealextreme which has R820T tuner:

dpavlin@blue:~$ sudo rtl_test -t
Using device 0: ezcap USB 2.0 DVB-T/DAB/FM dongle
Found Rafael Micro R820T tuner

It's a good choice, shipping is free (but takes few weeks), and it works well with ADS-B which was first interesting application which I wanted to try.
I also ordered E4000+2832U USB DVB-T TV Receiver Stick for $16.50 but it hasn't arrived yet so I don't have any experience with it.

I also have a good fortune that my friend is very skillful in radio communications, so he created variant of Coaxial Collinear Antenna for ADS-B Receiver. This one is more sensitive to static electricity which means it's not suitable for top of skyscraper scenarios, but in works very nicely. Under construction is filter which would cut everything expect interesting part of RF spectrum which would also solve problem of static electricity.

On the picture, you can see part of it together with output in browser created using dump1090. It's important to note that normal DVB-T antena without any modification will work with R820T tuner since it's most sensitive so far, but range will be much smaller.

We also had good luck with getting data about keypresses from power-socket controlling device using rtl_433 from github with just a peace of wire as antenna for short range tests.

munin migration and upgrade to v2

I always loved graphs. For my monitoring needs I'm using munin so in last few days I was migrating one installation from machine behind DSL line to co-location (virtual) server. Using old munin server, I would occasionally get e-mail alerts about unavailable services, mostly because n2n (which we use to connect to nodes) forgot about node or dropped packets because of busy DSL line. In the process, I decided to jump ahead and install version 2.0-rc5 from Debian unstable. This blog post will try to describe my journey...

I started simply by installing new munin instance in new (virtual) machine. I decided that historic data is important, so I decided to move configuration and graphs over to new instance. Simply coping rrd files over didn't went all that well and resulted in dreaded This RRD was created on another architecture error. This was unfortunate but rrd files where so large, that transfer won't fit into 5 minute munin poll interval anyway, so I had to take different approach.

To keep all historical data and not loose any munin polls while I transfer them I decided to first configure new munin node to poll all clients (so new updates will be preserved) and while this is running copy over rrd files from old server. This involved editing all nodes (9 of them!) and Cluster SSH came as perfect solution to add additional allow ^192\.168\.0\.10$ lines in /etc/munin/munin-node.conf on all nodes.

Coping rrd files had to be done using rrdtool dump/restore and it had to be done over compressed ssh link due to slow DSL line. For that, small shell script came very handy:

#!/bin/sh -x

dir=/var/lib/munin/maxxo


via_ssh="ssh -o ControlPath=/tmp/ssh_sock -o ControlMaster=auto -o Compression=yes root@10.1.3.10"

if [ ! -e /tmp/ssh_sock ] ; then
        $via_ssh read
fi

ls $dir/*.rrd | while read file
do
        echo $file
        rrdtool dump $file | $via_ssh rrdtool restore - $file --force-overwrite
done

You need to start it twice. First invocation will ask for password and open master ssh socket which next invocation will use for transfers of rrd files using compressed ssh link, without connection overhead for each file. We are talking about 4560 rrd files with total of over 250Mb, after all... Even with all this, it took hour and a half to transfer all that over, so setting up update of existing files was really required.

You might think that it's all, but unfortunately, it's not. Looking in /var/log/munin/munin-update.log I could see [FATAL] Socket read timed out to node. Terminating process.. Some of nodes required more time than default value provided by munin (30 sec) to respond with all data. It seems that ipmi plugins are notoriously snow to respond for example. To change server-side timeout, you have to pass --timeout 90 to munin-update utility. Unfortunately, in Debian you can't do that by modifying munin-cron invocation in /etc/cron.d/munin because it passes all parameters to munin-limit which doesn't have timeout option and dies on you (moral of the story: check cron e-mail while configuring cron jobs). In the end, I edited /usr/bin/munin-cron directly, changing one line:

/usr/share/munin/munin-update --timeout 90 $@ || exit 1

This will probably break with next update, but this blog post will remind me to change it again :-)

There where also a few tweaks on munin-node plugins to make them work inside kvm. iostat_ios plugin from munin-plugins-extra didn't like virtio devices which have major number 254, same as LVM2 devices which it ignores. Following patch solved this problem:

diff --git a/cs-munin/etc/munin/plugins/iostat_ios b/cs-munin/etc/munin/plugins/iostat_ios
index 1380eb1..823df63 100755
--- a/cs-munin/etc/munin/plugins/iostat_ios
+++ b/cs-munin/etc/munin/plugins/iostat_ios
@@ -101,7 +101,7 @@ sub filter {
         return 0 if ($major ==   1); # RAM devices
         return 0 if ($major ==   9); # MD devices
         return 0 if ($major ==  58); # LVM devices
-        return 0 if ($major == 254); # LVM2 devices
+        return 1 if ($major == 254); # LVM2 devices and KVM virtio
     }
     if(defined($tmpnam)) {
         return 0 if ($tmpnam =~ /part\d+$/);

I also decided to use ksm which is enabled by following line in /etc/rc.local:

echo 1 > /sys/kernel/mm/ksm/run

And of course, now I had to graph it with simple shell ksm munin plugin. Dropping sharing line on this graph makes me think that it wasn't really needed, but we'll see in few more days.

To track other kvm parameters, I used munin-libvirt-plugins which comes with it's own helper script munin-libvirt-plugins-detect which you have to run to enable plugin and generate configuration.

For a web server, I opted to use apache2 and libapache2-mod-fastcgi together with graph_strategy cgi and html_strategy cgi in /etc/munin/munin.conf mostly to save some performance on polling machine. To make it work, I had to copy /etc/munin/apache.conf into /etc/apache2/conf.d/munin and uncomment relevant fast-cgi lines inside. After that, dynamically generated html is available at http://munin.example.com/munin-cgi/ and if you ever run munin-html before, you will still get old (obsolete) html pages if you visit page directly.

Next step would probably be to get rrdcached up and running...

Update: If you 404 or 500 errors when generating cgi graphs, and you will have to add cgitmpdir /tmp/munin/ to /etc/munin/munin.conf.

Fix Arduino Mega 2560 avr-libc issue

 
If you have Arduino Mega 2560 on any updated Linux distro with avr-libc 1.8 you probably have an issue with compiling.
 

Compiler usually fails with this error: attempt to use poisoned “SIG_USART0_RECV”
 
Quick fix is to downgrade to earlier version of avr-libc…
 
Grab older version of avr-libc from Koji:
wget http://kojipkgs.fedoraproject.org//packages/avr-libc/1.7.1/4.fc18/noarch/avr-libc-1.7.1-4.fc18.noarch.rpm
 
Now delete problematic avr-libc 1.8:
sudo rpm -e avr-libc --nodeps
 
And finally install avr-libc 1.7:
sudo yum localinstall avr-libc-1.7.1-3.fc17.noarch.rpm
 
Done.
 

Novi službeni laptop

 
Eto već su prošle godine i stari nettop koji me par godina služio je odradio svoje, i ako se zvijezde poklope ima šanse da dobijem novi laptop, budget nije neograničen i moram birati između dva branda HP i Lenovo.
 
Prvo sam mislio da ću birati između nekog i3 laptopa no kako su cijene dosta pale, već se za razumnu svotu može dobiti i5, a neki modeli čak imaju i SSD disk!
 
Prvo treba krenuti od zahtjeva…

laptop treba biti prijenosan, težina oko 2 kg, nikako preko 2,2 kg

ekran 13-14″ opet zbog lakšeg nošenja okolo

SSD, ako ikako može, no cijena je tu faktor

mat ekran

kvaliteta izrade

 

Googlajući razne recenzije izenandio sam se kako čak ima dosta laptopa po pristojnoj cijeni. No pozivam sve vas da mi date neki svoj favorit ili javite koji se vama od modela koje sam stavio u uži izbor najviše sviđa.
 
 
 

Lenovo IdeaPad U310 59-353968 i5-3317U

Baterija: oko 5h-6h

Težina: 1.7 kg

 
ZA: 13.3″ ekran, 24GB SSD + 500 GB HDD, lagano otvaranje, i5 CPU
PROTIV: bijela boja, cijena 5700 kn, glare ekran, tipkovnica se uvija na pritisak?, vrući zrak iz tipkovnice kod gaminga?
 
 

HP-Pavilion-dm4-3000

i5 2450M + Intel HD Graphics 3000 + ATI 7470M GPU

Baterija: 6-cell, 55Wh, 3.5h gledanjve filma do čak 7.5h

Težina: 1.97 kg

 
ZA: i5 CPU, backlight tipkovnica, 14″ ekran, cijena od 5200 kn, dobar zvuk, baterija, fensi-šmensi
PROTIV: ATI GPU s upitnom Linux podrškom, glossy ekran, nema SSD
 
 

Lenovo ThinkPad Edge E430

Intel Core i5 3210M + Intel HD Graphics 4000

Baterija: 6-cell, 62Wh, do 6.5h

Težina: 2.15 kg

 
ZA: i5 CPU, antiglare ekran, Intel HD Graphics 4000, 14″ ekran, dimenzije, jači CPU od prva dva modela
PROTIV: nema SSD

 
 

Mega pušiona

Viđao sam po Twitteru kako neki ljudi citiraju Kim Dotcoma i podržavaju njegovu borbu za slobodu informacija te prenose kojim mjerama bi on smanjio piratstvo. Sve lijepo i krasno, ali kad to dolazi od osobe s bogatom kriminalnom prošlošću onda je jasno da su te njegove riječi samo maska za nešto drugo.

Što je zapravo Kim želio postići s Megom? Pretpostavljam da je projektni zadatak glasio napraviti file sharing servis kojeg će ljudi masovno koristiti zbog piratskog materijala, a koji će pravilima korištenja nas osigurati od tužbi. I to je upravo ono što je s Megom napravio.

Na prvoj stranici se širi natpis THE PRIVACY COMPANY dok većina ljudi to ispravno čita THE PIRACY COMPANY. U pravilima, koje ste prihvatili ako koristite servis, lijepo piše da oni nisu odgovorni za sadržaj kojeg ljudi dijele, a složili ste se da ćete Kimu i njegovoj kompaniji platiti sve moguće pravne troškove koje bi mogli imati ako ih netko tuži zbog sadržaja kojeg ste postavili na njihov servis.

Ali podaci su kriptirani i nitko bez ključa ne može saznati njihov sadržaj, reći će neki naivac koji vjeruje onome što Kim govori. Zbilja? Kako onda objašnjavate činjenicu da će oni obrisati datoteku koju ste uploadali ukoliko je istu već netko uploadao? Zar vaši podaci ne bi trebali biti enkriptirani? Zaboravite to. Njima je u interesu da smanje trošak hostinga i ovo je prva stvar koju će svaki file sharing hosting napraviti kako bi smanjio troškove. Cijela ta priča s enkriptiranjem sadržaja je samo mazanje očiju kako bi se stvorio privid da vas nitko neće kazniti zbog piratiziranja jer kao ne može se saznati što ste poslali na Megu. Kim se hvali da samo 0,001 datoteka na Megi krši autorska prava. Kako to zna kad ne zna što je u njima?

Ne zavaravajte se, Mega je servis jednog muljatora i prevaranta koji pleše na rubu zakona. On je sebe osigurao i spreman vas je žrtvovati, sami ste pristali na to.

Nginx with Apache and suPHP on cPanel

Recently I was testing nginx as a proxy for Apache and PHP on cPanel installation. I came a cross nginxadmin plugin for cPanel, By default Apache is configured in WHM to run PHP in SuPHP mode. After installing nginxadmin I wanted to test index.php on my test website but I got error message saying “500 Internal Server Error”. Index.php is just calling phpinfo() function.

Looking at the error_log revealed the following:

[Sat Jan 19 14:12:06 2013] [error] [client 192.168.1.200] SoftException in Application.cpp:256: File “/home/user/public_html/index.php” is writeable by group
[Sat Jan 19 14:12:06 2013] [error] [client 192.168.1.200] Premature end of script headers: index.php

So the solution is to make chmod 640 on /home/user/public_html/index.php file or to tweak SuPHP settings to allow files with group writable permissions.
If you don’t want to change file permissions on every .php file you can change the setting in /opt/suphp/etc/suphp.conf file. Find the line ‘allow_file_group_writeable=false‘ and change the parameter value to true.

Benchmark flash memory on Linux

 
I gathered all my flash based memory devices (USB flash memory sticks and SHDC cards) and wanted to see how do they compare to each other.
 

Quick google search revealed that dd tool could be used for benchmarking so that is what I used. Have you done benchamrs some other way? Please share your method.
 

Be really, really careful if you choose do to write test, because that will ERASE your existing data so test write speed only on EMPTY flash memory devices. Also triple check which device is your flash memory device. Simple typo could render your data partition unusable!
 

I didn’t want to write on the first 1GB of flash memory because that part is probably most often used, I know there are some wear-leveling chips that should take care of that but you can use skip parameter if you wish to be safe. To prefent mesuring your cache performance but actual device performance use iflag and oflag options. 
 

read speed benchmark:

$ sudo dd if=/dev/sdX1 of=/dev/zero bs=1M count=400 iflag=direct

write speed benchmark :

$ sudo dd if=/dev/zero of=/dev/sdX1 bs=1M count=400 skip=1000 oflag=direct

table.tftable {font-size:12px;color:#333333;width:100%;border-width: 1px;border-color: #729ea5;border-collapse: collapse;} table.tftable th {font-size:12px;background-color:#acc8cc;border-width: 1px;padding: 8px;border-style: solid;border-color: #729ea5;text-align:left;} table.tftable tr {background-color:#d4e3e5;} table.tftable td {font-size:12px;border-width: 1px;padding: 8px;border-style: solid;border-color: #729ea5;}

SDHC Card	Read speed	Write speed
Adata 8GB class 10	20.8 MB/s	10.6 MB/s
Apacer 4GB class 4	15.4 MB/s	3.6 MB/s
LG 2GB class unknown	20.9 MB/s	8.1 MB/s

table.tftable {font-size:12px;color:#333333;width:100%;border-width: 1px;border-color: #729ea5;border-collapse: collapse;} table.tftable th {font-size:12px;background-color:#acc8cc;border-width: 1px;padding: 8px;border-style: solid;border-color: #729ea5;text-align:left;} table.tftable tr {background-color:#d4e3e5;} table.tftable td {font-size:12px;border-width: 1px;padding: 8px;border-style: solid;border-color: #729ea5;}

USB memory stick	Read speed	Write speed
2GB Transceng VF V90	19.2 MB/s	11.6 MB/s
4GB Kingston DT 108	15.2 MB/s	7.2 MB/s
8GB Kingston DT 2.0	19.0 MB/s	16.0 MB/s
8GB Kingston DT SE9	20.1 MB/s	14.9 MB/s
4GB Red USB	19.8 MB/s	9.0 MB/s
8GB Silicon Power SP 810	24.8 MB/s	5 MB/s

Restarting

This week it came to my mind it's been more than a half of a year since my last post. Be assured, it was for a reason. This was probably the most exciting half of a year ever in my life, so let's start from the beginning.

So what changed since then? Long story put short - I got married, I relocated from Croatia to California, and got my first Silicon Valley job. After interviewing and examining my options for a really long time (partly my fault, partly due to a really long interviewing process) with Google and then Facebook, I finally got an offer from Facebook to work in their HQ in Menlo Park, CA as a production/infractructure engineer. The position described was really awesome, basically a great hacking job involving programming, system administration and design, scaling etc. Basically a devops position. I relocated to San Francisco in late September.

Given the opportunity to work on some of the world's largest linux deployments, writing a lot of Python code and debugging some of the hardest engineering problems, coupled with opportunity to work with the top IT talent around and living in geek paradise is really exciting, and I'm sure I'll have much to write about.

Ima li Ubuntu for phones ikakve šanse?

Kad sam vidio najavu na ubuntu.com stranici prvo što sam pomislio je da će glavna tema biti Ubuntu for Android jer on je realno ostvariv u Canonical okvirima (jasno od prve inačice i ne bi očekivali da radi baš kako treba, ali…). Ali Mark i ekipa zagrizli su u veći kolač odlučili se za Ubuntu for phones. Nisu išli od nule već su preuzeli ono što se preuzeti može a to su Android kernel i driveri, najkritičniji dio. Dalvik su zanemarili i odlučili da umjesto virtualne mašine koriste nativne aplikacije. Nisu zanemarili niti HTML5 i mogućnost pokretanja web aplikacija.

Kako će se razvijati aplikacije za Ubuntu for phones možete vidjeti i probati već i danas, Go mobile stranica ima kratki uvod u razvoj uz pomoć Ubuntu QML toolkita koji se bazira na Qt okruženju.

Canonical kombinira postojeće open source tehnologije, a sve misli zamotati u svoj UI (koji bi trebao biti skalabilan ovisno o veličini i tipu zaslona) i na to dodati svoje servise (Ubuntu One).

Ubuntu for phones bi mogao imati šanse kad bi Canonical što prije otvorio platformu i omogućio developerima da na postojeće uređaje instaliraju njihov OS ta da se s njime mogu početi igrati. Ako je vjerovati filteru na GSMArena stranici trenutno postoji 219 uređaja, raspoloživih na tržištu, koji zadovoljavaju kriterije za entry level Ubuntu smartphone. I to bi trebao napraviti što prije. Ako će Canonical taktizirati, čekati s predstavljanjem dok ne bude imao prvi pravi uređaj, stvari na tržištu bi se mogle promijeniti. Uz cijelu paletu postojećih platformi Samsung gura još i Tizen, RIM bi s BlackBerry 10 platformom mogao biti veliko iznenađenje jer već sad spominju ugovore s 50-ak operatera koji su već navikli na dobru suradnju s njima. Dok se pojavi novi uređaj iduće godine Ubuntu bi mogao pripremati ulet na stražnja vrata rabljenih uređaja čiji su korisnici razočarani jer u pravilu nemaju nadogradnje na novi Android. Korisnička iskustva i povratne informacije bi im pomogle da premijera bude što bolja. Pozitivna iskustva korisnika mogla bi natjerati i proizvođače da prije prihvate novi OS.

Canonical nema novaca niti mašineriju velikih igrača. Microsoft je utukao silne novce u promociju svoje mobilne platforme i nisam siguran da su baš zadovoljni postignutim. Canonicalova jedina i najveća šansa su korisnici. Samo nekako u zadnje vrijeme ne pokazuju da znaju sa zajednicom.

P.S. Ne znam na kojim komponentama Androida Microsoft uspijeva unovčiti svoje patente, ali ako bi se eliminacijom Dalvika i postojećeg UI-a mogao suziti taj manevarski prostor onda bi proizvođačima mogao biti puno zanimljiviji.

libvirt vnc console without virt-manager

We have all been in this situation. We need quick access to hardware console of machine to debug nasty hang (kernel oops in my example). But now-days, we are all in the cloud, and that machine is libvirt's kvm. I just need access to VNC console, and installing whole virt-manager seems like overkill. So, let's see if we can make it somewhat easier and quicker.

First ssh into your host machine (garak in this example, tunneling some VNC ports along the way:

dpavlin@x200:~$ ssh $(seq 5900 5904 | awk '{ print "-L "$1":localhost:"$1 }') garak

If you have more than 4 vms running on this machine, increase 5904 above.

Then, using virtsh list all domains, and examine vnc port:

virsh # list
 Id    Name                           State
----------------------------------------------------
 1     stargate                       running
 2     treca                          running
 15    hroug                          running
 16    close                          running

virsh # vncdisplay close
:2

Now, back on laptop, in another terminal start vnc viewer:

dpavlin@x200:~$ vncviewer :2

Since close didn't respond to reboot, reset or shutdown commands in virsh, I opted to kill kvm process and start it again with start close. Quick alternative to virt-manager installation.

Is wireless keyboard safe for your passwords?

I recently got big screen TV (big for my living room at least). It came with few HDMI ports and VGA, so next logical step was to connect computer to it. And of course, then I noticed that it would be really handy to have wireless keyboard and mouse to complete this nice setup. However, I also wanted to ssh over that networks, so I started examining how secure wireless keyboards are. tl;dr; summary: they are not secure.

First I asked for suggestions which wireless keyboard to buy. I have quite big fingers, so mini models just doesn't do it for me. I got suggestion to take a look at Logitech K400 and sure enough it seemed like good choice. One of first things that I noticed is that it supports 128-bit AES encryption. I started to have a good feeling about it, but I wanted to know more, so I hoped to Logitech Advanced 2.4 GHz Technology pdf and discovered that not all keys are encrypted.To quote documentation:

The encryption applies to all standard keyboard keys (a, s, d, f...) and modifiers (Shift, Ctrl, Alt...). The multimedia keys (Play, Pause, Mute...) that may be implemented in some keyboards are transmitted in clear text.

How can I trust keyboard which doesn't encrypt all traffic? This got me thinking. Can I somehow verify that keys are encrypted? Is this wide-spread problem? Can I make mistake and broadcast my keystrokes to whole world?

Sure I can. For older 27Mhz keyboards there is KeyKeriki v1.0 - 27MHz project which implement sniffer for it (video DeepSec 2009: Keykeriki: Universal Wireless Keyboard Sniffing For The Masses). But, Logitech is 2.4Ghz, so it's secure, right? Well, there is KeyKeriki v2.0 - 2.4GHz which does same for 2.4Ghz (video Keykeriki V2 - Practical Exploitation of Modern Wireless Devices [SIGINT10]). OK, Logitech does some kind of AES on top of that, but since it does transfer some keys unencrypted, and it's proprietary technology I can't really check that.

I also got suggestion to use bluetooth keyboard because it's secure. Well, quick search revealed Ubertooth One which basically defeats bluetooth protection with a bit of sniffing and a little brute force.

By this point, I was puzzled. Is there secure wireless keyboard with touchpad which I can buy? Something I can be sure that it encrypts all traffic as opposed to only some keys? Or is usb extension cable only real solution for me?

Hybrid IRCD for Arch Linux

Hybrid IRCD has been a favorite of mine for many years. I tried it once because a Croatian IRC network ran it and it stuck with me. I'm very happy to announce Hybrid packages for Arch Linux are available in AUR from today. I worked on it as a side project for a while and finished today thanks to the blizzard that kept me inside this weekend. Hybrid server is available as ircd-hybrid, and Hybserv2 services are available as ircd-hybrid-serv. They adhere to standards set by all other ircd providers, default configuration for both is usable out of the box, and examples for connecting services to the server are included. They were built and tested on both arches, only component not tested by me are systemd service files.

GNU/Linux and ThinkPad T420

I got a new workstation last month, a 14" laptop from the ThinkPad T series. The complete guide for TuxMobil about installing Arch Linux on it is here.

It replaced a (thicker and heavier) 13" HP ProBook 4320s which I used a little over a year, before giving up on it. In some ways ProBook was excellent, certified for SUSE Linux it had complete Linux support down to the most insignificant hardware components. In other ways it was the worst laptop I ever used. That ProBook series has chiclet-style keyboards, and I had no idea just how horrible they can be. Completely flat keys, widely spread and with bad feedback caused me a lot of wrist pain. Even after a year I never got used to the keyboard, and I was making a lot of typos, on average I would miss-type even my login every second boot. At the most basic level my job can be described as a "typist" so all this is just plain unacceptable.

The touchpad however is worse than the keyboard. It's a "clickpad", with one big surface serving as both the touchpad area and the button area. To get it in a usable state a number of patches are needed, coupled with an extensive user-space configuration. But even after a year of tweaking it was never just right. The most basic of operations like selecting text, dragging windows or pressing the middle button is an exercise in patience. Sadly clickpads are present in a huge number of laptops today.

Compared to the excellent UltraNav device in the ThinkPad they are worlds apart. Same is true of the keyboard in T420, which is simply the best laptop keyboard I've ever used. I stand behind these words as I just ordered another T420, for personal use. One could say these laptops are in different categories, but that's not entirely true. I had to avoid the latest ThinkPad models because of the chiclet-style keyboards they now have. Lenovo is claiming that's "keyboard evolution", to me they just seem cheaper to produce, and this machine could be the last ThinkPad I'll ever own. If this trend continues I don't know where to turn next for decent professional grade hardware.

Twitter doesn’t scale (for conversations)

Note: this post is about people, not technology.

Let’s imagine for a moment that you’re following someone on Twitter that’s fairly popular in your niche – not a star, mind you, but someone with a couple thousand followers. Someone like Gary Bernhardt or Phil Sturgeon.

One fine day, the person you follow tweets something interesting, thought or discussion provoking, or perhaps incorrect, and you find yourself wanting to reply.

Now for the trick question: do you reply?

If you do, you might be one of tens (or hundreds) of people saying the same thing, effectively flooding and annoying the target. But if you don’t, your possibly unique and interesting feedback won’t get heard. It’s not that unique, you decide, and you weren’t asked directly, why bother…

So you can either be apathetic or troll. Your choice.

What if you’re on the receiving side – you’re the big fish and get hundreds of replies to every other tweet you send? You can either ignore them (except from the people you know and maybe follow yourself), effectively making Twitter a broadcast medium ala RSS, or actually try to strike conversation with all the people replying to you, wasting a lot of time.

So you either use Twitter as RSS (with a few notable exceptions) or waste a lot of time and probably get annoyed a lot to boot. Your choice.

Beyond a few thousand followers, Twitter just doesn’t scale for conversations. You can either reach a big crowd or actually talk to a small number of people, but not both.

Blogs, Facebook or Google+ actually don’t have this problem. Why? Because, when commenting on someone’s post, you can see all the other comments. It’s up to you to decide whether you’re contributing something meaningful or meetoo-ing.

On Twitter, there’s no way to do that.

WebCamp Zagreb 2012

Last Saturday I participated in a WebCamp Zagreb conference. The conference was organised by several web developers groups from Zagreb (I was one of the organisers on behalf of Croatian Python community), with participants from entire Croatia and wider – we had speakers as well as attendees coming from Slovenia and Serbia as well. I’m also proud to say my company Good Code was one of the sponsors of the event.

The atmosphere was phenomenal and the conference itself was extremely well received. Actually we were limited by the venue size, since the interest both from the speakers (over 60 really good talk proposals for 24 slots) and atttendees (all the free tickets were “sold-out” in hours, and by my estimate, there were around 250 people in total). And the biggest complaint we had is that we hadn’t printed out the schedule for people who aren’t WiFi-enabled. Yeah, stupid mistake that, but I’m happy that was the biggest one :)

I was one of the presenters, and I talked about our experiences with introducing code reviews, unit testing and continuous integration - my slides are available on Speakerdeck. Don’t worry if you don’t understand Croatian, they just served as background for my talk anyways. All the talks were recorded and hopefully the videos will be ready in a few weeks. The talk was well received and I got a lot of feedback and questions from the audience (and even more later on the post-conference drinkup event). Although I’m confessedly not an automated-testing/continuous-integration expert, it seems that our from-the-trenches knowledge in starting down that path was very interesting to the audience.

My colleague Goran talked about lessons learnt from a project we did in Good Code – a distributed feature film video management system (slides on speakerdeck, will update the post when videos are available).

All in all, a great event. Can’t wait for the next years’, which is bound to be even bigger and better :-)

 

WebCamp Zagreb - konferencija visokog ritma

WebCamp Zagreb 2012 je bila jednodnevna konferencija web developera koju su zajednički organizirale korisničke grupe čiji članovi bi nekada, u drugim prilikama, vodili svete ratove pokušavajući dokazati da je njihova tehnologija bolja. Web se unatrag par godina pokazuje kao vezivno tkivo, kao zajednički problem kojeg treba riješiti, a rješenja se često ne ograničavaju samo na jednu tehnologiju već se kombinira više rješenja u pokušaju da se problem riješi na najbolji način.

Od najave konferencije do puštanja karata u prodaju prošlo je vrlo malo vremena, a prva runda ulaznica je planula za 7 sati. Konačni broj sudionika je došao do broja 240. Program se odvijao u dvije dvorane i često se bilo vrlo teško odlučiti kuda krenuti.

Prva stvar koju bih želio istaknuti je trajanje predavanja. 25 minuta je odlično pogođeno vrijeme, ni prekratko, ni predugo, a predavači su bili primorani da se usredotoče na ono najvažnije. Organizatori su se domislili odlične funkcije: čuvara predavača. Dotični nije imao ulogu da čuva predavača od razuzdane gomile već je pazio i upozoravao da predavač ne prekorači termin. To je odlično funkcioniralo pa je i zadnje predavanje počelo točno na vrijeme. Često se na konferencijama drže onog klasičnog školskog sata od 45 minuta, ali na njima je i disperzija pažnje veća jer malo koji predavač može držati visoki ritam cijelo to vrijeme. Preporučio bih organizatorima da se i idući put odluče na isti vremenski raspored jer ovo je savršeno funkcioniralo. Ovo je valjda prva konferencija na kojoj nisam markirao nijedan termin. Dinamično, informativno, bez praznog hoda.

Predavači su bili odlični. Najbolja predavanja su bila ona gdje su se iznosili konkretni primjeri, iskustva i preporuke. Tako bi trebalo nastaviti i ubuduće. Ljude ne zanimaju predavanje gdje se iznose neke osnovne informacije koje je moguće doznati posjetom jednoj web stranici. Ljude zanimaju konkretna iskustva, priče “eto mi to tako radimo”. Na kraju krajeva zbog takvih stvari ljudi i dolaze na konferencije. Samo što negdje drugdje to možda doznaju u direktnom kontaktu, negdje na hodniku, a ovdje smo to imali prilike čuti direktno na predavanju.

Kad se spremate na neko developersko predavanje prva stvar koju očekujete vidjeti je kod. WebCamp je imao nekoliko odličnih predavanja u kojima je kod bio u drugom planu, a više se govorilo kako razviti neki proizvod, komunicirati s ljudima (Maratz i Merlin), kako organizirati rad tima (Luka i Tomislav) ili na koji način organizirati distribuirani sustav (Goran). Odličan je bio i Saša s Erlangom u praksi i nije mi žao što me je punjenje mobitela spriječilo u namjeri da markiram predavanje.

Goran Peretin je svojim nastupom pokazao i kako može bez problema nastaviti s karijerom IT stand-up komičara jednog dana kad prestane pisati dobar kod.

Neka vam bude žao ako niste bili. Ja s nestrpljenjem čekam snimke predavanja da vidim one koje sam morao propustiti. Organizatori su odlično odradili svoj posao pa ne sumnjam da ćemo vrlo brzo biti u prilici to pogledati.

Printrbot experience - 3D printers are not for everyone

Allmost a year ago, me and three other friends decided it's a very good idea to support Printrbot kickstarter project and get our-selfs 3D printer. We didn't have any particular use for it (other than printing Raspberry Pi case when it arrives) but it seemed like the right thing to do. This post will try to explain how far did we manage to get with it and why we where wrong.

If you examine original Kickstarter page you will see following description:

A desktop 3D printer you can build in a couple hours. Print plastic parts you design or download - even parts for another printer.

Our experience can't be further from that statement. For a start, Brook Drumm (to whom I'm ethereally grateful for his efforts to make 3D printers commonplace) got his campaign funded with 1,808 backers who spent $830,827 instead of just $25,000 goal he envisioned. This was both good and bad. Good part was that after funding we knew that we will have 3D printer (I'm carefully not mentioning printing anything), but the bad part was logistics: there was simply no way he would be able to print 1808 versions of original design on 3D printers themselves (idea of RapRap-like printers, which Printrbot was one iteration, was always to make them self-replicating). So, he decided to change design and move toward wooden laster-cut parts for most of construction, and print just parts which where necessary.

This also introduced significant delay in printer shipment, but when you are funding Kickstarter project, you should be prepared for it, so I'm not complaining. When it finally arrived this summer (10 months after end of Kickstarter campaign), it was significantly bigger than I expected:

To be honest, we did upgrade to bigger Printrbot PLUS so I guess we should expect a lot of parts. As we are mostly software geeks, we did only reasonable thing to do: check if all parts are present comparing it with bill of materials which we got printed out.

This is the point where our problems started. We had missing one bag of parts which included termistor and switches. We contacted Printrbot HQ and they sent us missing parts. We started assembling following Printrbot Building Instructions by Brook Drumm and it took us more than 50 hours to get to our first blob.

Of course, it didn't work perfectly on first run. We where trying to print 5mm Calibration Cube Steps is ABS plastic which we received with our Printrbot (we even got additional 450g of ABS plastic as replacement for power supply which wasn't part of international shipments).

Actually, it still doesn't work well as you can see in video below, but we are hopeful. In the meantime we figure out that best source of information is Printrbot Talk forum and wiki. Forum is somewhat depressive since most users have some kind of problems with their built, just as we do.

To be honest, we didn't expect smooth ride. However, as I mentioned before we are not really hardware hackers, and my only conclusion is that home-made 3D printers are really for people who already have enough experience to make their own 3D printer, and not for software guys like us. However, we won't give up, and I fully expect to have working printer (after we get replacement barrings from Printrbot HQ because our are sticky). We are collecting useful add-on models and instructions on our Printrbot wiki page but I didn't expect that we will have to contact Printrbot HQ twice for missing and replacement parts. But eventually we will be able to print Raspberry Pi box, I hope :-)

Quo vadis, Ubuntu?

Tamo negdje prije osam godina, nakon izlaska prve inačice Ubuntu distribucije, napisao sam post u kojem sam ispravno predvidio (požderite se vi profići iz Gartnera i sličnih prodavaonica magli koji prognoziraju ono što im se plati) da će ta distribucija postati jedan od glavnijih igračna na Linux sceni. Jednostavno, svi sastojci za dobar Linux desktop bili su tu.

Izdanja s oznakom LTS bila su odlična za računala koja dugo niste željeli reinstalirati. Baš sam neki dan zamijenio 10.04 na jednom starijem laptopu. Čini mi se da je isti došao s 8.04 inačicom (DELL je na svoje laptope stavljao Ubuntu), predložio sam vlasnici da ne gazi, tu za nju čudnu biljčicu, piratskom inačicom Windowsa. I danas mi je zahvalna i s gnušanjem odbija prijeći na OS kojeg vrte svi ostali članovi njezine velike obitelji. Razlog je jednostavan, nema problema s računalom već se može koncentrirati na ono za što joj služi računalo, na rad.

Sada na na njemu vrti Ubuntu 12.10. Nekada bi, nakon završetka instalacije, prva stvar koju bih napravio na novoinstaliranom računalu bila apt-get install ubuntu-restricted-extras. Vremena su se promijenila. Sada je prva stvar koju radim dodavanje ppa:gwendal-lebihan-dev/cinnamon-stable u izvore paketa. Dao sam šansu Unityju, ali stvari predugo stoje na mjestu i Cinnamon je puno bolje rješenje.

Unity je donio par dobrih i inovativnih rješenja, ali ostvarenje je loše, a cijeli projekt pati od bolesti koje ne bi očekivali od projekta otvorenog koda. Ono što meni od prvog dana nedostaje je mogućnost podešavanja, prilagođavanja. Nije me mučio ni nedostatak sučelja već činjenica da nema jednostavne, čovjeku čitljive datoteke za konfiguraciju. Ono što je najgore ni danas stvari nisu ništa bolje, za bilo kakve promjene morate imati instalirane nekakve alate treće strane. Indikacija da nešto nije u redu je nedostatak proširenja za Unity (osim raznih leća). Nije prihvaćen od developera i za to postoje opravdani razlozi.

Zajednica je blagoslov i prokletstvo za otvoreni kod. Po onoj staroj da puno babica znači kilavo dijete. Mark Shuttleworth je tu odigrao odličnu ulogu (osim financijske) jer je donosio odluke, a često open source projekti zapnu zbog toga jer odgađaju donošenje ključnih odluka jer se zajednica ne može dogovoriti.

Zadnje dvije godine očito je da Mark donosi pogrešne odluke. Čemu Ubuntu na tabletima i pametnim telefonima? Umjesto da se fokusiraju na razvoj kvalitetnog desktop OS-a, oni ga optimiziraju za uređaje na kojima teško da će uhvatiti. 14.04 LTS bi trebala biti ta kritična inačica. Sumnjam da će biti. Ono što mu nedostaje je podrška proizvođača tih uređaja, oni trebaju biti spremni da na njih stave Ubuntu. Možete li naći barem jedan jeftini kineski uređaj na kojem je Ubuntu? Druga važna stvar su aplikacije. Klasične desktop aplikacije uglavnom ne funkcioniraju na uređajima na dodir tako da neće biti prevelike koristi od postojećeg repozitorija. Ono što bi moglo proći je Ubuntu for Android, ali ne vidim neki pomak u tom projektu.

12.10 inačica prilikom skidanja vam nudi priliku za donaciju. Linux korisnici su spremni donirati i kupovati, ali samo ako točno znaju što kupuju. Mark bi trebao priznati da je možda pogriješio i među stavke za donaciju ubaciti i ‘Pokopaj Unity’. Neki bi ljudi platili samo za to.

mdadm - how to tune your md software RAID

We had software RAID in form of md devices on Linux for a long time. Here are few useful hints how to make it work better...

Scrub your array to check data consistency

I will assume that you are already monitoring your disks using smart, but from time to time it's useful to force full re-scan of your array to make sure that all data is still there and consistent. Some filesystems provide this option to scrub data on it's own (zfs and btrfs comes to mind) but if your filesystem is located on md array you can always force it using

echo check > /sys/block/md0/md/sync_action

I would suggest to do this from cron, hopefully during weekend or some other time when your load is lower.

Adding write-intent bitmap to speed up recovery

If you installed your md array a long time ago, you probably didn't turn on write-intent bitmap. It's very useful when you have to recover because bitmap will track changes and this will prevent long re-sync times when disks have to read and compare every block. To turn it on use:

mdadm --grow --bitmap=internal /dev/md0

Mirror between two devices of same speed

Recently, one of my 500Gb disks in RAID1 (mirror) failed. I decided to replace it with 1Gb drive which was unfortunately green drive (which basically means slow). Adding two drives of different speed in mirror will reduce performance to single slower drive which is a shame. Since I wasn't able to add additional disk and wasn't prepared to give up redundancy of data I started searching around and found that I can specify one disk as write-mostly using:

mdadm --add /dev/md0 --write-mostly /dev/sdb1

Same trick will work on combination of hard drive and SSD, but in that case, you will slow down writes to speed of your hard drive.

Net-installing Arch Linux

Recently I had to figure out the most efficient way of net-installing Arch Linux on remote servers that fits into the deployment process, with many other operating systems, which runs a DHCP and TFTP daemons serving various operating system images.

The Arch Linux PXE wiki put me on the right track and I downloaded the archboot-x86_64 ISO, which I temporarily mounted, so I can copy the key parts of the image:

# wget http://mirrors.kernel.org/archlinux/iso/archboot/2012.06/archlinux-2012.06-1-archboot-x86_64.iso 
# mkdir /mnt/archiso
# mount -o loop,ro archlinux-2012.06-1-archboot-x86_64.iso /mnt/archiso

Let's say the TFTP daemon serves images using pxelinux, chrooted in /srv/tftpboot. The images are stored in the images/ sub-directory and the top level pxelinux.cfg configuration gets copied from the appropriate images/operating-system/ directory automatically based on the operating system selection in the provisioning tool:

# mkdir -p images/arch/arch-installer/amd64/
# cp -ar /mnt/archiso/boot/* images/arch/arch-installer/amd64/

The boot directory of the archboot ISO contains the kernel and initrd images, and a syslinux installation. I proceeded to create the pxelinux configuration to boot them, ignoring syslinux:

# cd images/arch/
# mkdir arch-installer/amd64/pxelinux.cfg/
# emacs arch-installer/amd64/pxelinux.cfg/default

  prompt 1
  timeout 1
  label linux
    kernel images/arch/arch-installer/amd64/vmlinuz_x86_64
    append initrd=images/arch/arch-installer/amd64/initramfs_x86_64.img gpt panic=60 vga=normal loglevel=3

# ln -s arch-installer/amd64/pxelinux.cfg ./pxelinux.cfg

To better visualize the end result, here's the final directory layout:

arch-installer/
arch-installer/amd64/
arch-installer/amd64/grub/*
arch-installer/amd64/pxelinux.cfg/
arch-installer/amd64/pxelinux.cfg/default
arch-installer/amd64/syslinux/*
arch-installer/amd64/initramfs_x86_64.img
arch-installer/amd64/vmlinuz_x86_64
arch-installer/amd64/vmlinuz_x86_64_lts
pxelinux.cfg/
pxelinux.cfg/default

I left the possibility of including i686 images in the future, but that is not likely ever to happen due to almost non-existent demand for this operating system on our servers. Because of that fact I didn't spend any time on further automation, like automated RAID assembly or package pre-selection. On the servers I deployed assembling big RAID arrays manually was tedious, but really nothing novel compared to dozens you have to rebuild or create every day.

From a fast mirror the base operating system installs from the Arch [core] repository in a few minutes, and included is support for a variety of boot loaders, with my favorite being syslinux which in Arch Linux has an excellent installer script "syslinux-install_update" with RAID auto detection. I also like the fact 2012.06-1 archboot ISO still includes the curses menu based installer, which was great for package selection, and the step where the base configuration files are listed for editing. Supposedly the latest desktop images now only have helper scripts for performing installations - but I wouldn't know for sure as I haven't booted an ISO in a long time, Arch is an operating system you install only once, the day you buy the workstation.

Another good thing purely from the deployment standpoint is the rolling releases nature, as the image can be used to install the latest version of the operating system at any time. Or at least until the systemd migration which might obsolete the image, but I dread that day for other reasons - I just don't see its place on servers, or our managed service with dozens of proprietary software distributions. But right now, we can deploy Arch Linux half way around the globe in 10 minutes, life is great.

Sve je remix (ili Povratak u off topic)

Danas sam pogledao Osvetnike. Zapravo i nisam pogledao cijeli film, možda pola, od početka, propustio sredinu, vratio se na kraj. Uf, stvarno je Hollywood zatupio publiku kad je taj film polučio tako dobro otvaranje i tako dobru zaradu. Ništa novo, ništa neočekivano, ništa što već nismo vidjeli stotine puta. Samo da se zna, ja sam kao klinac (a i malo kasnije) gutao Marvelove stripove i sigurno sam pogodno tlo da se film primi. Ali eto nije.

Večeras sam pogledao jedan drugi film koji mi je bio neusporedivo zanimljiviji i zabavniji: Frequently Asked Questions About Time Travel. U njemu jedan od glavnih junaka piše pismo koje započinje ovako…

Dear Hollywood, you’re shit. How hard can it be to make a film that doesn’t suck?

Britanska komedija o putovanju kroz vrijeme, čija se radnja odvija samo u jednom pubu, bolja je od holivudskog blockbustera? To ne samo da nije iznenađenje, to je zapravo pravilo. Čovjek koji je napisao scenarij je i redditor, a optužen je i za scenarij jedne epizode serije Dirk Gently.

FAQATT završava obradom pjesme The Final Countdown koja mi se odmah dopala više od originala: Eskimo Disco - The Final Countdown. Negdje u komentarima neki rimatoe (po svemu sudeći iz .ba) je natipkao…

just watched FAQ! great remix

Remix?! Da, pravi je trenutak da se spomene Kirby Ferguson i njegov govor Embrace the remix. I jedan i drugi film su stvoreni miješanjem već postojećih elemenata. Samo što, kao i kod klasičnih glazbenih remixeva, u jednom slučaju imate vrhunsku produkciju i dosadnu stvar dok je u drugom slučaju sirova produkcija dovela da osvježavajućeg pića. Meni ni Matrix nikada nije bio veliko otkriće kao drugima jer sam u njemu prepoznao previše elemenata koji su na bolji način bili obrađeni negdje drugdje (u SF literaturi).

Kirby govori o Appleu i o pušta Stevena Jobsa koji kaže kako dobri umjetnici kopiraju, a veliki kradu. Par godina kasnije razbijesnio ga je Google i kaže kako će uništiti Android jer je to ukradeni proizvod. I ovih dana vidimo kako Samsung mora platiti milijardu dolara jer je kopirao Appleove patente. Prije 16 godina Jobs je rekao…

And we have, you know, always been shameless about stealing great ideas.

Obično licemjerje. Oni su mogli krasti od drugih, ali drugima neće dopustiti da kradu od njih.

Činjenica je da ništa nije potpuno nova inovacija, ništa nije potpuni original. Sve ima svoje ishodište u onome što je inovator tokom vremena naučio od drugih ljudi. Softverski patenti su problematični jer ne štite inovacije već ih zatiru. Što bi se dogodilo da je tada veliki Xerox zatukao mali Apple? Oni jesu izmislili veliki dio elemenata sučelja koje danas koristimo, ali nisu znali napraviti korak dalje. Apple je to znao i napravio jer je to tada bilo moguće.

Veliki imaju tešku artiljeriju patenata, sreća je za male što je tu artiljeriju preskupo trošiti na njih. Ali čim malo provire prijeti im paljba.

Hollywoodu je sve daleko lakše. Ako im se dopadne neki film oni će napraviti lošiju kopiju i nazvati to remake. Nemaju oni patente…

More on Redis

In managed hosting you're not often present in design stages of new applications and sometimes you end up supporting strange infrastructure. Or at least that was my experience in the past. So little by little I found my self supporting huge (persistent) Redis databases, against my better judgment.

Someone sent me a link to the Redis Sentinel beta announcement last month. It may even make it into the 2.6 release... but all of this I had to implement on my own long ago. A lot of developers I supported didn't even want to use the 2.4 branch (in my opinion just the memory fragmentation improvements are more than enough reason to ditch 2.2 forever). Another highly anticipated Redis feature, the Redis Cluster, may not even make it into the 2.6 release. That's too bad, there's too much features with Redis that are always "just around the corner", yet I have a feeling I'll be supporting Redis 2.4 for at least another 3 years, with all its flaws and shortcomings (I scratched the surface in my last article with AOF corruption, and not-so-cheap hardware needed for reliable persistent storage).

Typically I would split members of a Redis cluster across 2 or more power sources and switches. But that's just common sense for any HA setup, as is not keeping all your masters together. Redis doesn't have multi-master replication so a backup 'master' is always passive, and is just another slave of the primary with slaves of its own. If the primary master fails only half of the slaves have to be failed-over to the backup master. This has its problems (ie. double complexity of replication monitoring by the fail-over agents), but the benefits outweight failing-over a whole cluster to the new master. That could take half a day, as fail-over is an expensive operation (it is a full re-sync from the master). You can find replication implementation details here.

If you can't allow slaves to serve stale data (tunable in redis.conf) you need enough redundancy in the pool to be able to run at half capacity for at least a few hours, until at least one of the outdated slaves is fully re-synced to its new master. And that finally brings me to knowing when is the right time to fail-over.

Any half decent load balancer can export the status of a backend pool through an API, or just a HTTP page (if yours can't it's time to use the open source HAproxy). That information is ripe for exploiting to our advantage, but we need to be weary of false positives. I can't share my own solutions, but you will want all N slaves confirming that the master pool is truly degraded, and initiate fail-overs one by one to avoid harmonics if you are serving stale data, or all at once if you aren't. For all that you will need them to communicate with each other, and a simple message broker can do the job well.

As I am writing these last notes I realized I haven't mentioned another fundamental part of any Redis deployment I do - backups. This article documents the persistence implementation in Redis, and explains that the RDB engine snapshots are good for taking backups. RDB is never modified directly, and snapshots are renamed into their final destination atomically only when they are complete. From here it's trivial to write a script that initiates a background save, waits until it's done and transfers the fresh snapshot off site.

MATE desktop with Compiz Fusion on Fedora 16/17

 
If you are running Fedora 16 or 17 and you are missing your old GNOME desktop with Compiz Fusion effects then you will love this post.
 
Thanks to the great work of Wolfgang Ulbrich for his awesome work on packaging MATE Dekstop for Fedora 16/17 and even bringing it to Fedora 18 by default!
 
Installing MATE desktop is very easy on both Fedora 16 or 17, but installing Compiz Fusion on Fedora 16 is a bit tricky.
 

Fedora 16:

yum install https://dl.dropbox.com/u/49862637/Mate-desktop/mate-desktop-fedora-updates/16/noarch/mate-desktop-release-16-6.fc16.noarch.rpm

yum groupinstall MATE-Desktop

 
Fedora 17:

yum install https://dl.dropbox.com/u/49862637/Mate-desktop/fedora_17/mate-desktop-fedora-updates/noarch/mate-desktop-release-17-2.fc17.noarch.rpm

yum groupinstall MATE-Desktop

 
After sucessfuly installing MATE desktop just log out and on login screen after clicking on your username click on “session” and choose “MATE” before typing in your user password.
 
Installing Compiz Fusion is easier on Fedora 17, but if you follow these steps you will install it without issues on Fedora 16 also.
 
Compiz Fusion on Fedora 16:

yum install -y yum-plugin-versionlock nano

 

nano / etc/yum/pluginconf.d/versionlock.list

 
Now you need to add this text to file “versionlock.list”:

ccsm-0.8.4-5.fc16.noarch
libcompizconfig-0.8.8-1.fc16.x86_64
libcompizconfig-devel-0.8.8-1.fc16.x86_64
compizconfig-python-0.8.4-3.fc16.x86_64
compiz-plugins-main-devel-0.8.8-1.fc16.x86_64
compiz-mate-0.8.8-1.fc16.x86_64
compiz-manager-0.6.0-14.fc16.noarch
compiz-plugins-extra-mate-0.8.8-1.fc16.x86_64
compiz-plugins-unsupported-devel-0.8.8-1.fc16.x86_64
compiz-plugins-extra-0.8.8-1.fc16.x86_64
compiz-plugins-main-mate-0.8.8-1.fc16.x86_64
compiz-plugins-unsupported-0.8.8-1.fc16.x86_64
compiz-bcop-0.8.8-2.fc16.noarch
compiz-0.8.8-1.fc16.x86_64
compiz-plugins-extra-devel-0.8.8-1.fc16.x86_64
compiz-plugins-unsupported-mate-0.8.8-1.fc16.x86_64
compiz-plugins-main-0.8.8-1.fc16.x86_64
compizconfig-backend-mateconf-0.8.8-1.fc16.x86_64
compiz-devel-0.8.8-1.fc16.x86_64
compiz-gtk-0.8.8-1.fc16.x86_64
fusion-icon-0.1.0-0.8.5e2dc9git.fc16.noarch
fusion-icon-gtk-0.1.0-0.8.5e2dc9git.fc16.noarch

 
After you close and save “versionlock.list” you can install Compiz Fusion packages:
 

yum install --enablerepo=mate-desktop-fedora-new-application-testing compiz compiz-mate fusion-icon-gtk compiz-plugins-main compiz-plugins-extra compiz-plugins-extra-mate compiz-plugins-main-mate compiz-plugins-unsupported compiz-plugins-unsupported-mate

 
Compiz Fusion on Fedora 17:

yum install –enablerepo=mate-desktop-fedora-new-application-testing compiz compiz-mate fusion-icon-gtk compiz-plugins-main compiz-plugins-extra compiz-plugins-extra-mate compiz-plugins-main-mate compiz-plugins-unsupported compiz-plugins-unsupported-mate

After all packages are installed just start “fusion-icon” command and enjoy your “new” MATE deskop with Compiz Fusion ;)
 

cli oneliner – sort top 10 directories

 
One simple oneliner that I picked up few years ago, and it is great for finding directories that eat up most of space in current directory.
 
ls -A | grep -v -e '^\.\.$' |xargs -i du -ks {} |sort -rn |head -11 | awk '{print $2}' | xargs -i du -hs {}
 

Internet Archive BookReader plack server

Last year, I had good fortune to get acquiented with great work which Open Library does. It's part of Internet Archive which itself is a library. So, libraries are not (yet) dead it seems. Brewster Kahle's Long Now Talk explains it much better than I can do, so take 90 minutes to listen to it.

Most interesting part of Open Library (IMHO) is Internet Archive BookReader which is JavaScript application which allows users to browse scanned books on-line. For quite some time, I wanted to install something similar to provide web access to our collection of scanned documents. I have found instructions for serving IA like books from own cluster, but I didn't have a cluster, and converting all documents to IA book format seemed like an overhead which I would like to avoid.

Instead, I decided to write image server for JavaScript front-end using plack. I mean, it's basically a directory with images, right? Oh, how wrong can I be? :-)

It turs out that we have pictures in multiple formats (so sorting them required removing common prefix and using number only to get correct order), and most of are scanned images in pdf documents. Here are all types of documents which can be automatically collected into book for on-line browsing:

• images of scanned pages
• multi-file pdf file with single image per page
• single pdf file with one image for each page
• single pdf file with more than one (usually 4) horizontal bitmap strips for each page
• normal pdf documents which contain text and needs rendering to bitmap

Source code of my plack server for Internet Archive book reader is on github, so if you want to take a look, hop over there...

Easy VirtualBox VM management for the command line

I tend to create a virtual machine for each of the projects I’m working on. I’m using VirtualBox, which has nice enough UI, as well as a set of commandline tools I can use. Since I create and destroy my VMs quite often, I wanted to streamline this a bit.

To that end, I created a single template virtual machine (out of Debian Sid), set it up the way I like:

• set up an account with typical shell environment, and with my SSH key add to authorized hosts so I don’t have to type password to log into it
• install a few packages I use most of the time: git, ipython, vim-nox, …
• set up shared folders so I don’t have scp/rsync stuff all the time
• set up port forwarding for SSH and HTTP traffic

Then, for each new machine I want to create, I simply clone this machine and I’m up and running in a few seconds. I start the machine, work in it, and then suspend it so I can pick up where I left off when I come back to it later.

This can all be done quite easily with the VirtualBox tools, but as I’m starting/stopping/creating the machines all the time, I built myself a handy shell script to simplify it:

$ vm list
Available virtual machines:
"DebianServerTemplate" {83113c2e-22f7-46fa-bcdd-7b1a641ace9b}
"anothervm" {aee1140c-fe4b-4146-bc93-34ce08eaf5cd}
"projectvm" {3b361f0a-d34a-41cc-82b8-cb734c465ec1}
$ vm create testmachine
Create a new virtual machine named 'testmachine' [y/N]?
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Machine has been successfully cloned as "testmachine"
$ vm start testmachine
Waiting for VM "testmachine" to power on...
VM "testmachine" has been successfully started.
$ vm status
Running virtual machines:
"testmachine" {fe48fd07-d2d2-471c-b191-89aa873cab7c}
$ vm stop testmachine
"testmachine" {fe48fd07-d2d2-471c-b191-89aa873cab7c}
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
$ vm destroy testmachine
No way Jose. If you're really sure, destroy it via the GUI

The omission of VM destroying in the shell is intentional – I don’t want to accidentially destroy a VM due to my fingers being quicker than my brain.

The script is available here. You’ll want to edit it to use your VM template UID (change the BASE_TEMPLATE variable in the script), as it’s going to differ from mine.

Comments, suggestions, bugreports, patches are welcome. And before you mention Vagrant – Vagrant is cool, but this is way faster.

Premladi za umiranje, prestari za programiranje

Kako i dalje neki neupućeni misle da je programiranje posao koji je stvoren za rane dvadesete i kako nitko pametan ne programira poslije tridesete odlučio sam napraviti malu anketu da vidim što o tome misle programeri na društvenim mrežama (Twitter, Facebook grupa Developers Hrvatska). Anketu je trenutno ispunilo 117 91, 70 programer i rezultati su sljedeći…

Koliko imaš godina?

• 26-35 60% 57%, 64%
• 19-25 23% 27%, 25%
• 36-50 14% 12%, 10%
• <=18 2% 1%
• >51 2% 1%

Iznad zamišljene idealne granice je 76% 70%, 74% programera. Znači nisu svi sagorjeli.

Koliko još misliš programirati?

• Do kraja života, dok me prsti i mozak služe 48% 46%
• Još 10 godina 17% 18%, 19%
• Još 5 godina 12% 11%, 14%
• Još 15-20 godina 9% 11%, 9%
• Dok ne nađem drugi posao 9% 8%, 7%
• Do mirovine 5% 7%, 6%

Dugoročno misli programirati 53% 61% programera, a kratki fitilj (5 godina i do drugog posla) ima 21% 19%, 21%programera dok je na sredini nekih 26% 29%, 19% programera.

Da li se programer može umoriti od programiranja?

• Nekad da, nekad ne 54% 55%, 51%
• Da, to je posao na kojem se brzo potrošiš 21% 23%, 26%
• Programiranje je način života, možete li se umoriti od života? 20% 21%
• Ne 5% 2%

Tek nešto manje više od četvrtine ispitanih smatra da je programiranje posao na kojem se čovjek brzo potroši.

Učenje novih programskih jezika, tehnologija i metoda ti je…

• …jako zanimljivo. 38% 40%, 36%
• …veselje i radost svaki novi dan. 25% 31%
• …izazov kojeg uspijevaš savladati. 16%
• …u opisu posla i moraš ga prihvatiti. 15% 13%
• …nužno zlo. 5% $%, 3%
• …izuzetno naporno. 1%

Kao glavni razlog zbog čega je posao programera naporan neki navode činjenicu da treba stalno učiti nove stvari. Tu se već vidi neupućenost onoga koji to tvrdi jer se u informatici dosta toga vrti u krug. Neukima će se učiniti da je svaki put nešto novo, ali oni koji imaju više iskustva će vidjeti da se stvari ponavljaju. Čak 63% 65%, 67% ispitanika u učenje novih stvari nalazi vrlo zanimljivim i to ih veseli. Tek 6% 5%, 4% smatra to zlom i napornom aktivnošću. 31% 29%ih je negdje na sredini. Tu su neuki koji pišu o programerima najviše promašili. Učenje novih stvari nije takav bauk kakvim ga oni smatraju.

Ispitanici su navodili i 5 najdražih programskih jezika. Neki su naveli samo jedan, većina ih je navela 5 jezika, neki nešto manje, a poneki i više. Najpopularniji među ispitanicima je Python, onda su na vrlo kratkom razmaku JavaScript, C#, PHP i C++, dok su C i Java dobili 20 glasova, a Ruby je na kraju grupe popularnih s 13 glasova. Samo jedan glas su dobili Ada, D, Go, Lua i Vala.

P.S. Prekrižene vrijednosti su iz vremena prve i druge objave rezultata (ukoliko je bilo promjena). Zadnja promjena 25.6.2012.

Samsung Galaxy S2 vs Ubuntu PC performance

Introduction

It seems that many people assume that 1.2 GHz dual core mobile ARM CPU should be almost as fast as a PC CPU running on a similar frequency. They're wrong.

ARM cores are indeed more power efficient per square mm of surface on a same production process than Intel x86 and AMD64 architecture processors. Most of the efficiency comes from a simpler and more space efficient instruction set, but that advantage typically benefits only front-end of the CPU, which is not the biggest spender of those precious miliwatts.

The other reasons why modern dual or quad core mobile phones can run on a fraction of power that notebook or desktop (PC) CPUs need:

• less computation units on CPU die (less SIMD, ALU, etc units)
• smaller cache than PC CPUs
• power gating parts of CPU (but laptop and desktop CPUs also do this for a number of years)
• significantly slower DRAM interface  than PC CPUs, using slower DDR RAM (LPDDR2)

So how much slower is your Android cell phone RAM than your PC RAM?

Measuring time, pythonic approach

Function wrapping with Python decorators, and code wrapping with Python context managers (i.e. context manager __enter__() and __exit__() functions) can easily be used for measuring time used for running a function or code snippet. Here is the simple implementation of both approaches:

import sys
import time

if sys.platform == 'win32':
  timer = time.clock
else:
  timer = time.time


# use as decorator
def measure_time(func):
  def _measure_time(*args, **kwargs):
    start_time = timer()
    res = func(*args, **kwargs)
    print 'Time elapsed for {0}: {1}'.format(func.__name__, timer() - start_time)
    return res
  return _measure_time


# use as context manager
class MeasureTime:
  def __enter__(self):
    self.start_time = timer()

  def __exit__(self, exception_type, exception_val, exception_traceback):
    print 'Time elapsed: {0}'.format(timer() - self.start_time)

Usage case for both approaches:

>>> @measure_time
... def test():
...     time.sleep(2)
...
>>> test()
Time elapsed for test: 2.00204205513

>>> with MeasureTime():
...     time.sleep(2)
...
Time elapsed: 2.00203204155

tc-play is free (as in freedom) version of TrueCrypt

 
TrueCrypt is free (as in beer) software used for on-the-fly encryption of data, but it is not licenced under open source licence.
tc-play is free (as in freedom) tool whic provides full support for creating and opening/mapping TrueCrypt-compatible volumes and it is licenced under GPL licence.
 
Because TrueCrypt software is under a poor license, which is not only non-free, but has the potential to be actively dangerous to end users or distributors who agree to it, opening them to possible legal action even if they abide by all of the licensing terms, depending on the intent of the upstream copyright holder. Fedora continues to make efforts to try to work with the TrueCrypt upstream to fix all of the issues in their license so that it can be considered Free, but have not yet been successful. – taken from Fedora Forbidden Items page.
 
There has been no articles regarding tc-play, and that seams strange since it provides great functionality, hopefully some reviews and tutorials will soon follow.
 

DORS/CLUC 2012 conference

As you all know by now, last week we had another DORS/CLUC conference. This time I had two talks and one workshop.

Sysadmin cookbook

I never proparly introduced this project here, but if you want to know more about my convention based documentation examine the presentation of hop over to http://sysadmin-cookbook.rot13.org/ and take a look at generated documentation.

Basic idea is to document changes in easy to write files on file system (preserving symlinks to files on system which allows you to quickly see if cookbook is deployed or not and diff between template and deployed configuration). I know that my cookbook is mix of various things I did in last three years, but I do find it useful, so hopefully it might be useful to you also.

Kindle - so much more than ebook reader

This was longer talk about my one year experience with Kindle. I must say that I'm still very happy user of Kindle, but in this talk, I tried to cover Kindle Developer's Corner at mobileread forum as well as other related projects:

• Free Libre command-line tools for Kindle development
• A PDF (plus DJVU, ePub, TXT, CHM, FB2, HTML...) viewer made for e-ink framebuffer devices, using muPDF, djvulibre, crengine mobileread forum thread
• A VNC viewer for ebook readers which allows you to use Kindle as second screen mobileread forum thread

So if you are still wondering if it's worth the effort to install third-party software on Kindle, answer is yes, it is!.

Web scale monitoring

This was a workshop which doesn't have much with web (it's about monitoring ADSL CPE devices and provider equipment in-between), but it shows (I hope) nice way to integrate several project to provide nice scalable monitoring infrastructure. It's composed of:

• Gearman message queue together with Gearman::Driver provide on-demand scaling of workers
• redis saves all data from external systems (LDAP, CRM) and all results from statistics collection nicely providing data for web interface
• PostgreSQL stores all collected data, using hstore to provide unstructured key value store for different data from different devices while still allowing us to use SQL to query data (and export it to data warehouse)
• Mojolicious provides web interface which uses data from redis and provides JSONP REST interface for Angular.js

All in all it's a nice combination of tools which served my need quite well, so I hope it was also useful to people who attended workshop.

lm-sensors automatic deployment, issue with sensors-detect

 
If you wish to deploy lm-sensors on multiple diferent linux machines issue is with sensors-detect what ask you quite a few questions.
 
I asked on IRC and got few suggestions. First one is if you wish to accept defaults which sensors-detect suggest you need just to press [ENTER] all the time. To automate this use this one liner:
 
(while :; do echo ""; done ) | sensors-detect
 
If you wish to override defaults and answer YES to all questions then use this oneliner:
 
yes "" | sensors-detect
 

Maximum wifi speed with OpenWRT? (802.11a Turbo mode aka Super A)

 
Hi, we have been testing what is the absolute maximum speed over wifi when using openwrt, for out test we used this hardware:
 
- 2 x Ubiquity Router station
- 2 x Wistron CM9 802.11a/b/g mini pci cards
- 2 x 3dBi omni antennas
 
- openwrt version – backfire 10.03.1
- wifi driver – madwifi
- distance – 2 meters (same room)
- encryption – psk2
 
For benchmarking we installed iperf package and used default settings (tcp packets in one direction).
 
First we tested “plain” 802.11a mode (hwmode 11a) on channel 40 and we got 30-35Mbit speeds.
 
Then we switched to 801.11a turbo (hwmode 11ast) and got 55-60 Mbit speeds.
 
Is this the maximum speed possible? Are there any other tweaks we can do to get faster speeds?
 
Did you get faster speeds? How? Which gear did you use? We didn’t make results artificially better by using udp packags, my guess by using udp packages we could get near 100 Mbit speeds.
 
Are there any openwrt tutorials for tweaking and benchmarking maximum speed?
 

rpcbind is new portmap or how to make nfs secure

I was installing NFS server on otherwise public host recently, and noticed that conventional wisdom about securing NFS server is somewhat dated. My goal was to expose NFS on two internal interfaces without exposing it to whole wide Internet (assumptions about network security changed a lot since NFS was designed, sadly).

For a start, you are probably running rpcbind instead of portmap on recent Debian installations. So you will need to modify flags which are passed to portmap on startup:

root@rsync1:~# cat /etc/default/rpcbind 
OPTIONS="-w -l -h 172.16.10.2 -h 192.168.0.219"

You will also need to add following line:

root@rsync1:~# grep rpcbind /etc/hosts.deny 
rpcbind: ALL

Now you will notice that rpcinfo -p still works OK on localhost. That's because rpcbind will always add loopback address, so we have to test it from another machine:

root@rsync1-dev:~# rpcinfo -p 192.168.0.219
rpcinfo: can't contact portmapper: RPC: Authentication error; why = Client credential too weak

That's more like it! If we take a look in log...

root@rsync1:~# tail -1 /var/log/auth.log
May  8 20:31:51 rsync1 rpcbind: connect from 192.168.0.21 to dump(): request from unauthorized host

...we don't even have to guess local system IP adress. We'll allow this host to connect with...

root@rsync1:~# grep rpcbind /etc/hosts.allow 
rpcbind: 192.168.0.21

We can also check our tcp wrappers configuration with:

root@rsync1:~# tcpdmatch rpcbind 192.168.0.21
client:   address  192.168.0.21
server:   process  rpcbind
access:   granted

Jadni naši developeri

Oleg Von Bug u svojoj kolumni u Mreži tvrdi kako:

developerski posao, koji nam se tada činio najboljim i najpametnijim na svijetu, nije bogznašto. Spada svakako u donji segment s računalima vezanih zanimanja današnjice.

Djelomično je u pravu, nekada sam običavao tvrditi kako su developeri najveći rudari među informatičarima. Svi drugi informatičari su puno ležernije dolazili na posao jer rješavanje skoro svakog problema najprije je krenulo od developera. U velikim sistemima obično bi ih doživljavali kao potrošnu robu. Neki bi ih voljeli vrijeđati i nazivati običnim koderima.

Stvari su se malo izmijenile zadnjih godina. Ljudi sve više vremena provode na webu, sve više se bave aplikacijama na svojim mobilnim pametnjakovićima, a aplikacije za tu ciljnu skupinu pojedinci i male skupine agilnih developera mogu puno brže i efikasnije napraviti od ogromnih timova u kojima su developeri samo kotačići na koje se gleda olegovski.

Startupmanija trese svijet, ima tu napuhanih balona, ali činjenica je da je ovo zlatno vrijeme za developere. Oni mogu najbrže unovčiti svoje sposobnosti i stvoriti nove vrijednosti. Kad se pogledaju najbrže rastuće domaće informatičke tvrtke onda vidimo da su ih redom osnovali developeri, proizvod njihovog rada im je glavna roba i bez problema se probijaju na strana tržišta.

Hm, ako developerima, kao donjem segmentu, može krenuti tako dobro, gdje li je onda gornji segment? Oni su već otplovili prema zvijezdama?

Oleg na pijedestal podiže svoje znanje asemblera za 6510 procesor i zbog jednog maturalnog rada misli da može govoriti o developerskom poslu. Ne Oleže, to ne znači da si developer. Ako se nijedna tvoja aplikacija nije srela s konačnim sudom koji se zove Korisnici u produkciji onda se ne možeš nazvati developerom. Ti boluješ od sindroma i ja sam nekada programirao koji se vrlo često javlja među onima koji su se s mikroračunalima sreli osamdesetih godina i počeli su učiti programirati jer je to bila jedna od vrlo rijetkih korisnih stvari koja se je mogla raditi na njima (a trebalo je i opravdati nabavu te skupe stvarčice pred roditeljima), ali nikada nisu završili u tim vodama jer su našli lakši posao. Kako ljudi u godinama vole pametovati tako i sindromaši vole započeti priču eh kad sam ja programirao, to je bilo programiranje, a ne ovo klikanje.

Pogledaju li se trendovi plaća informatičara u SAD-u onda se može vidjeti da index za developere konstantno raste dok za administratore ili project managere variraju i ostaju isti. Prosječna plaća developera je veća nego za spomenute druge dvije kategorije. A novinar može očekivati tek trećinu plaće koju će dobiti developer.

Od pravog novinara očekujem da se posluži činjenicima umjesto što vjeruje svojem sindromu iz prošlog tisućljeća.

Enable ad-hoc wifi networking on Android

 
Google has disabled wireless ad-hoc networking in Android from day one. Underlying Android libraries and wifi stack support adhoc networking and so far Google has provided no explanation why it still, even after four years, leaves adhoc disabled.
 
There has been huge outcry from Android users over this issue, but there is still no answer from Google Android developers.
 
There are few suggestons on xda-developers forum, and also on Stackoverflow, but most solution so far consisted of using new rom that had ad-hoc support enabled or manually messing about system config files.
 
But wait, there is an app for that!
 

There is an app called WiFi Ad Hoc enabler that should work on most Android devices.
 
If this app works for you, please give it a good review or notify developer of any issues you encounter, but consider that not all Android devices are supported yet.
 
I would be very grateful if you would vote up my answer on Stackexchange.
 
[1] https://code.google.com/p/android/issues/detail?id=82
[2] http://forum.xda-developers.com/showthread.php?t=754961
[3] http://stackoverflow.com/questions/1932150/can-android-do-peer-to-peer-ad-hoc-networking
[4] https://play.google.com/store/apps/details?id=nl.arendmedia.wifiadhocenabler
[5] http://www.arenddeboer.com/wifi-ad-hoc-enabler-for-android/
[6] http://stackoverflow.com/a/10108812/1031514

Kako napraviti Instagram i prodati ga Facebooku?

Oni developeri kojima je ego veći od sposobnosti će zacijelo reći da taj Instagram i nije nešto posebno i da bi to oni napravili za tjedan dana. O tome piše i Don Dodge u članku I could build Instagram in a week. Zaista, kad jednom netko nešto napravi, kad pokaže put, onda se čini da je sve tako jednostavno. Najbolja rješenja su najjednostavnija, ali malo tko će uspjeti doći do tog jednostavnog rješenja jer će biti zadovoljan onim prvim koje radi ili ne zna stvari rješavati na jednostavan način.

Ali ponekad za uspjeh čak ni tehnologija nije važna. Najbolji primjer je Twitter. Njegovi microblog konkurentni su skoro svi redom imali veće mogućnosti (Pownce je jedan od najboljih primjera), ali na strani Twittera je bila sreća i okolnosti koje su se poklopile baš u tom određenom trenutku.

Karakteristično i za Twitter i za Instagram je da rade samo jednu stvar dovoljno dobro i dovoljno jednostavno. Twitter je čak doslovno primjenio Unix filozofiju koja kaže:

Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface.

Instagram je napravio to isto, ali umjesto teksta omogućio je ljudima jednostavno izražavanje uz pomoć slika.

Ono što Don ne spominje u članku, a što smatram glavnim tajnim sastojkom koji je omogućio uspjeh spomenutih servisa je ograničenje. Twitter je ograničio poruku na 140 znakova i natjerao korisnike da se izražavaju jasno i precizno. Ta preciznost nije bila važna u prvo vrijeme kad su korisnici pisali što sada radim (neki još uvijek koriste Twitter na taj način) već kad su počeli pisati što se događa. Da je Twitter ukinuo to ograničenje i korisnicima dozvolio duže poruke ubrzo bi izgubio svoj smisao.

Osim ograničenja na broj filtera Instagram je korisnike ograničio formatom. Slika može biti bilo kojeg oblika sve dok želite da je u obliku kvadrata. Neki drugi slični servisi omogućuju rezanje slike na različite načine, stotine filtera i još puno raznih iživljavanja na slikama. Prekomplicirano. Korisnika treba ograničiti. Treba mu omogućiti da radi samo jednu stvar i ništa više od toga. Naravno da to nije primjenljivo na sve aplikacije, ali za slučaj kad imate hrpu korisnika pokazalo se kao dobar recept.

Klasična greška kod developera je da zanemaruju potrebe korisnika i njihovo mišljenje. Kod izrade aplikacije često mjere prema sebi, a ne prema korisniku. I još su skloni postavljati ograničenja koje im uvjetuje njihovo poznavanje neke tehnologije. Ako radite servis za korisnike onda oni moraju biti mjera. Mogu mogućnosti nekog grida biti fantastične, ali ako on ne ispunjava korisnikova očekivanja onda ste promašili s njim. Treba prepoznati pravu korisničku želju i ispuniti ju. Flickr je u početku bio alat za MMOG, a onda je započeo svoju evoluciju u današnji oblik.

Važno pravilo je i da nema pravila. Ne dozvolite da vas obeshrabre zbog toga jer takva aplikacija već postoji, zbog toga jer imate loš poslovni model ili nemate poslovni plan na više od 100 stranica. Važnije je da doslovno ne kopirate postojeća rješenje već pokušajte biti bolji ili barem drugačiji i jednostavniji. I oku ugodni i upotrebljivi. Nije loša taktika da nađete neki servis koji je dobar, rješava određeni problem, ali je prekompliciran za upotrebu. Napravite lakšu inačicu tog servisa, s manje mogućnosti, ali upotrebljiviju, intuitivniju (Pinterest je zapravo image bookmarking servis kakvih ima na bacanje, ali pogodio je zlatnu žicu). Ovo je vrijeme kad se traži jednostavnost. Zašto Microsoft uvodi Metro sučelje? Zašto Mark gura Unity u Ubuntu? Apple je oduvijek težio tome. Ljudi ne žele brdo mogućnosti koje neće koristiti. Žele jednostavne alate za pojedine poslove.

P.S. Don kaže da Instagram Facebooku vrijedi milijardu dolara. Ja se ne bih složio s njime. Facebook je debelo preplatio Instagram. Vrijeme će to sigurno pokazati.

When to do git merging or rebasing

This one pops up fairly often, and can indeed be quite confusing – when to use merge versus rebase in git? Here’s the answer.

The story so far …

In merge, you combine two divergent branches back into one. There’s also a special kind of merge called fast-forward, done when a branch being merged is just a continuation of the branch you’re merging into – so the new commits are just pasted on top of the target branch (ie. it is “fast-forwarded”).

In rebase, you change your branch (that’s being rebased) so it looks like it was branched off a new base, not the original one. This involves rewriting the commits, so you’ll end up with different commit IDs.

… and now the conclusion

So, when to use either? The rules are suprisingly simple:

Never rebase public branches

By “public branches” here I mean branches that other people might have checked out. Rebasing rewrites history, and anyone having branches that were checked out of the history you just unmade will be sad, angry or worse. That’s one reason you can’t just push rebased branch on GitHub (unless you force it and sacrifice a kitten). So just say no.

Rebasing private branches is perfectly fine, and in fact often done when squashing or rearranging commits, cleaning up a branch before going public with it, or just updating long-running feature branch (go easy on the last one, though).

Never merge upstream master into your master

By “merge” here I mean a recursive (non-fast-forward) merge, ie. the one that leaves tracks. And the rule is not only for master, but for any upstream tracked branch you may want to merge back into one day.

As a rule of thumb, fork master should always be in sync with upstream master (ie. only do fast forward merges/pulls), only having the commits that are in upstream already. Also, you should never have to merge master (local or remote) into a feature branch. If you need to update an existing feature branch, rebase it rather than merging into int.

Updating public feature branches

What to do if your public branch needs to be updated from upstream? You can’t rebase it since it’s public, and you can’t merge into it since you’ll want to merge it back some day.

Turns out it’s really simple: create a fresh one off of upstream, merge your branch into it, and continue working on a new one.

(This was originally a comment on the How To GitHub: A Complete Guide to Forking, Branching, Squashing and Pulls article Hacker News discussion. The article itself and the ensuing discussions are full of useful advice – if you haven’t already, go read them!).

Vook - da li je lako napraviti e-knjigu?!

Vook je servis koji bi vam trebao omogućiti jednostavno uređivanje i izdavanje elektroničkih knjiga. Jučer su svoj servis otvorili javnosti pa možete i sami isprobati kako je lako urediti vlastitu knjigu koristeći samo web preglednik. Najveći problem na tom putu je sadržaj, ali to je za neke pisce zanemariva sitnica. Iznos pretplate na njihov servis i nije tako zanemariv, da li je to opravdana investicija?!

Uređivanje je besplatno, a potreba za pretplatom počinje u trenutku kad želite objaviti svoj uradak. Ali sumnjam da će broj onih koji će doći do tog koraka biti dovoljan da opravda bombastične najave kao što su a game-changer, Call it iBooks Author without the marriage to Apple ili Could Vook (and similar ventures) be about to disrupt an entire industry?. Autori tih izjava sigurno nisu pokušali koristiti Vook.

Početak i ne izgleda tako loše, naslov, dodavanje poglavlja. Možete odabrati 4 osnovna stila koji pristojno izgledaju. Nisam se želio izložiti avanturi podizanja neke .doc (hm, na listi podržanih formata nema .rtf-a) datoteke već sam krenuo s prenošenje tekstova u Content editor. Tu su već počeli prvi problemi, editor se počeo svojeglavo ponašati, pa sam malo zavirio u kod. To nije ništa drugo nego CKEditor. Pokušao sam s ubacivanjem slike pa mi je mala slika od 200×150 točkica završila razvučena preko cijele stranice. Nema kontrole za rastezanje slika već morate ručno unositi dimenzije. Sučelje CKEditora je možda dobro za unošenje tekstova za web, ali kad trebate urediti veću količinu teksta nije niti praktičan ni upotrebljiv. Izgleda da se spomenuta jednostavnost odnosi na posao developera i implementaciju editora, a ne na rad s njim.

Nakon što sam unio par tekstova odlučio sam pogledati na što to liči. Odabrao sam generiranje ePub-a i dobiveni dokument pogledao u EPUBReaderu (dodatak za Firefox, bolji mi radi od FBREadera). Neki razmaci su se izgubili pa je tekst izgledao ružnije, prepoznao je samo prva dva podnaslova kod prikaza sadržaja, a ostali su zanemareni. Kako je ePub predviđen za e-čitače isprobao sam dokument na Pandigital Novel readeru (Android OS). Aldiko se mučio i prikazao samo naslovnu stranicu, AReader nije uspio učitati dokument, a Moon+ Reader se pokazao kao najbolji. No i on je prepoznao samo prva dva podnaslova, ali tekst se poprilično raspao i moj prvi pokušaj nije bio previše čitljiv. Eh da, ako vam netko kaže da je AReader najbolji ePub čitač na Androidu (a ta priča se širi) možete mu reći da nije u pravu i preporučiti Moon+ Reader.

I bez posebnih ePub validatora jasno je da Vook generira jako loš ePub dokument s dosta grešaka. To je glavni razlog zbog kojeg ne bi preporučio njegovo korištenje. Ima daleko boljih alata koji generiraju ispravne dokumente. Čak i dodatak za calibre koji iz rss-a stvara ePub radi to daleko bolje i točnije.

Vook nije opravdao ni moja minimalna očekivanja. Ako baš morate, probajte ga. Da vidite kako se to ne radi. Nije isključeno da će neki mazohisti napraviti nešto s njim. Ali bez puno peglanja to neće ići…

Infrastructure you can blog about

I spent last 5 months planning and building new infrastructure for one of the biggest websites out there. I was working around the clock while developers were rewriting the site, throwing away an ancient code base and replacing it with a modern framework. I found no new interesting topics to write about in that time being completely focused on the project, while the RSS feed of this journal was constantly the most requested resource on the web server. I'm sorry there was nothing new for you there. But I learned some valuable lessons during the project, and they might be interesting enough to write about. Everything I learned about Puppet, which was also a part of this project, I shared in my previous entry. I'll focus on other parts of the cluster this time.

Here's a somewhat simplified representation of the cluster:

Following the traffic path first thing you may ask your self is "why is Varnish Cache behind HAproxy?". Indeed placing it in front in order to serve as many clients as soon as possible is logical. Varnish Cache is good software, but often unstable (developers are very quick to fix bugs given a proper bug report, I must say). Varnish Cache plugins (so called vmods) are even more unstable, crashing varnish often and degrading cache efficiency. This is why HAproxy is imperative in front, to route around crashed instances. But it's the same old HAproxy that has proven it self balancing numerous high availability setups. Also, Varnish Cache as a load balancer is a nice try, but I won't be using it as such any time soon. Another thing you may ask is "how is Varnish Cache logging requests to Syslog when it has no Syslog support?". I found FIFOs work good enough - and remember traffic is enormous, so that says a lot.

Though with a more mature threaded implementation I can't see my self using Rsyslog over syslog-ng on big log servers in the near future. Hopefully threaded syslog-ng only gets better, resolving this dilemma for me for all times. Configuration of rsyslog feels awkward (though admittedly syslog-ng is not a joy to configure either). Version packaged in Debian stable has bugs, one of which made it impossible to apply different templates to different network sources. Which is a huge problem when it's going to be around for years. I had to resort to packaging my own, but ultimately dropped it completely for non threaded syslog-ng which is working pretty good.

Last thing worth sharing are Redis experiences. It's really good software (ie. as alternative to Memcached) but ultimately I feel disappointed with the replication implementation. Replication, with persistence engines in use, and with databases over 25GB in size is a nightmare to administrate. When a slave (re)connects to a master it initiates a SYNC which triggers a SAVE on the master, and a full resync is performed. This is an extremely expensive operation, and makes cluster wide automatic fail-over to a backup master very hard to implement right. I've also experienced AOF corruption which could not be detected by redis-check-aof. This makes BGREWRITEAOF jobs critical to execute regularly, but with big databases this is another extremely expensive operation, especially if performed under traffic. The following has proven it self as a best solution for high performing Redis servers; 4x147GB 15k SAS disks in (h/w) RAID10, and Xeon 5000 series CPUs.

While working on this the running joke was I'm building infrastructure you can blog about (but otherwise do little else with it). But it does do a little more than just look impressive on paper.

Read two-column pdf files on Kindle

I really like Kindle because it allows me to run my own selection of software on it. However, when I try to persuade other Kindle owners in all the benefits of running custom software, I usually get response like: but it already does everything I need. That might be true, but that's only because you never tried to read two column article on Kindle...

Fortunately, we have alternative pdf reader for Kindle based on muPDF library which is mostly written in lua. This is especially nice since it allowed me to add support for reading two-column pdf layouts which you can see in following video:

Basically, you press F to switch to new layout and then use fiveway buttons to move down the column, or right to move to top of next column, with page change if needed. If you move to left, you will be positioned to bottom of previous column which is useful if you want to read again last thing.

If you like this feature, liberate your kindle and download latest version of pdf reader, drop it in /mnt/us/customupdates and press Shift Shift I to install it from louchpad. Then press Shift P D to start it. Wiki pages of project describe all available shortcuts and there is active thread on mobileread forum.

HTML input placeholder handling with jQuery

There’s a million of small jQuery snippets which handle input placeholders (ie. the help/explanation text that’s shown until you actually write something in the field), and when HTML5 gets widespread, they’ll all be obsoleted.

But nevertheless, here’s another one.

this post is more for self reference, but if it’s useful to you, be my guest.

Printing from Koha to Zebra printers on local Windows machine

As you know by now, I'm somewhat biased towards pixel-exact printing on strange printers. This time around, I was tasked with requirement to make Koha print bar-code labels from web interface on Zebra S4M printers which are locally connected to Windows clients over USB.

At first sight, this seems like an easy task. Zebra printers are supported under CUPS on Linux and OSX, so there shouldn't be any problems, right? For a start I found out that CUPS driver doesn't work well, mostly because it's older that Windows version of driver, and doesn't seem to send all ZPL codes required to print label.
To make thing even worse, since printer is connected locally to Windows machine, it presents itself as Windows GDI printer which doesn't want to print ZPL (printer protocol) directly without wrapping ZPL in magic quotes and enabling it in Windows.

On the other side, Koha tries to print labels using normal print dialog in Windows. This won't work well, because we (again) need pixel exact label as opposed to web page randomly scaled to printer label. To make this worse, client Windows machines are behind firewall, so I can't send label to IP address of client because all I can see in request is IP address of our firewall.

To solve all this problem I decided to deploy following setup:

• Label design is done in Inkscape. To rasterize it to bitmap I decided to use rsvg-convert which has nice option to create exact bitmap size from SVG file in which I replace placeholders with variable values (bar-code and call number)
• Printers are installed in Windows using Generic/Text driver which is only one which will just pass data directly to printer
• To share printer to Linux, I decided to install lpd server so I don't have to open local Windows accounts to access printer (and they are behind firewall, so It's safe)
• To get local IP address of client (so I can send ZPL to internal IP address on which lpd is listening) I decided to redirect client to internal web server (behind firewall, same as clients) which knows client IP address, and knows where to send label.
• From user's perspective, Koha redirects clients to internal CGI script, which in turn rasterize label from request parameters, sends it to printer and redirect browser back to Koha page (with additional parameter of IP address of client/printer). This page shows label which was just printerd by pointing directly to internal server's PNG rendering of label. It also inserts data about printed label (including printer IP) in database for audit log.

Since I couldn't use CUPS to produce ZPL for printer, I wrote Printer-Zebra which can convert pbm and pnm formats (easily created from png label using pgntopnm). Even better, it also includes script which can render ZPL printer output back to pbm bitmap which is the only good way to verify that your solution doesn't anti-alias bar-codes or does something similar to reduce print quality on back and white printers. Rasterizer was also very useful when tracking differences between Windows driver output (gathered by printing to File on Windows) and CUPS one.

Snježni wifi kraljevi

Od danas Hrvatska uz natjecanje Snježna kraljica ima još jedno novo natjecanje a to su Snježni wifi kraljevi ;)
 
U natjecanju mogu sudjelovati svi zainteresirani, cilj je u što ekstremnijim uvjetima postaviti što više nove wifi opreme na krovove zgrada. Pravila su jednostavna a nagrade neprocjenjive.
 
Pogledajte kako je bilo u Osijeku prvom timi iz ekipe OsijekWirelessa koji su se okušali u ovome novome sportu na ovome online albumu.