Planet Linux.hr

BalCCon 2k19 - So, is Android a Linux?

November 04, 2019 15:21

Last weekend I had pleasure to attend BalCCon 2k19 and present my talk So, is Android a Linux? which is embedded below. It was great conference and I hope that my talk gave some food for though and hints how to run Linux on Android devices.

Emulate IR remote for TV or HVAC from command-line using Tasmota

August 17, 2019 17:04

I don't have TV remote. I did get one, but as soon as I installed TV I realized that it's quite annoying to find remote to turn TV on when I sit with my wireless keyboard (computer is the only device connected to TV). So, I added keyboard shortcut using xbindkeys, addad IR led to Raspberry Pi, configured lirc and was happy about it. And then, buster with kernel 4.19 came and everything changed.

Send IR to TV

Upgrade to 4.19 kernel should be easy, only thing you have to do (if your IR sending diode is on pin 18) is to enable new overlay:

# pwm works only on 18
dtoverlay=pwm-ir-tx,gpio_pin=18

This does not work reliably for me on Raspberry Pi 1. My TV detect roughly every third key press and this makes command-line TV remote solution useless because you can use TV menus to setup picture any more.

So, I had to do something. Getting up and pressing button on TV is not something that I can live with after having this automation working for year (and TV remote was missing by now). But, I had all required components.

Few weeks ago, I removed IR send/receive board from RMmini 3 and documented it's pinout:

I was also in the middle of flashing Sonoff-Tasmota to bunch of Tackin plugs so it seemed like logical step to flash Tasmota to NodeMCU board, connect RMmini 3 IR board to it and give it a try. And I'm glad I did.

I used to have http server (simple perl script) running on Raspberry Pi which used irsend to send IR codes. From xbindkey perspective, my configuration used curl and all I had to do to get IR working again was changing my script to use mosquitto instead of irsend:

mosquitto_pub -h rpi2 -q 2 -t cmnd/ir/IRSend -m '{"protocol": "NEC","bits": 32, "data": 0x20DF10EF}'

At this point I realized that I can put this into .xbindkeyrc and contact esp8266 directly. This didn't work... You can't have double quotes in commands which are executed and I had to put it into shell script and call that.

And to my amazement, there was noticeable difference in response time of TV. In retrospect, this seemed obvious because my TV nuc is much faster than Raspberry Pi, but this was probably the most unexpected benefit of this upgrade.

When I said that you have to connect IR receiver and sender on NodeMCU pins, you have to take care not to hit pins that have special purpose on power-up. For example, if you connect something that will pull to ground on powerup (IR led for example) to gpio0 esp8266 will stay in boot loader mode. gpio2 and gpio16 are led pins on nodemcu board, so don't use them (and define them as Led1i and Led2i in configuration).

Having LEDs configured in tasmota allows me to extend my shell script and blink led after IR code has been sent:

dpavlin@nuc:~$ cat tv-on.sh 
#!/bin/sh
mosquitto_pub -h rpi2 -q 2 -t cmnd/ir/IRSend -m '{"protocol": "NEC","bits": 32, "data": 0x20DF10EF}'
mosquitto_pub -h rpi2 -q 2 -t cmnd/ir/LedPower -m 1
mosquitto_pub -h rpi2 -q 2 -t cmnd/ir/LedPower -m 0

Send IR to HVAC

By pure luck, just a few days latter, my friend wanted to control his ACs from computer. Again tasmota came to the rescue. Since HVAC support in tasmota will increase firmware size over 512Kb (which breaks OTA upgrade on 1Mb modules) it's not compiled in by default. However, you can edit sonoff/my_user_config.h and uncomment it:

    #define USE_IR_HVAC                          // Support for HVAC systems using IR (+3k5 code)
    #define USE_IR_HVAC_TOSHIBA                  // Support IRhvac Toshiba protocol
    #define USE_IR_HVAC_MITSUBISHI               // Support IRhvac Mitsubischi protocol
    #define USE_IR_HVAC_LG                       // Support IRhvac LG protocol
    #define USE_IR_HVAC_FUJITSU                  // Support IRhvac Fujitsu protocol
    #define USE_IR_HVAC_MIDEA                    // Support IRhvac Midea/Komeco protocol

However, if you want to keep OTA update working, you will also have to turn off some other configuration options (I don't use Domoticz or Home Assistant) to keep firmware size below 512Kb.

To create IR sender, I decided to add IR LED, transistor and resistor to existing ESP-01 module with DHT11 board (which has 3.3v regulator on it) according to the following DaveCAD(tm) drawing:

If you are wondering why I'm connecting IR led to RX pin (gpio3), it's because gpio0 is special, gpio2 is already used for dht11 and TX (which is gpio1) is also special. Since we don't need serial, using single pin left RX saves the day. And this is the picture of the first prototype (on which I tried all pins until I settled on RX):

With all this in place and quick re-flash, we where than able to issue commands like this to control AC:

mosquitto_pub -h rpi2 -t 'cmnd/ir/irhvac' -m '{ "Vendor": "Mitsubishi", "Power": 1, "Mode":"Cold", "Temp": 25}'

mosquitto_pub -h rpi2 -t 'cmnd/ir/irhvac' -m '{ "Vendor": "Mitsubishi", "Power": 0}'

So, with all this, I hope that you don't have any excuse not to control your IR devices from a command-line.

Update: Just to make sure that you don't think this is my best soldering ever here is also picture of 4 more modules which will be distributed to my friends.

Samsung Galaxy S2 (ARM Phone) vs Ubuntu PC performance

July 30, 2019 12:32

Introduction

(this post has been updated in 2016)

It seems that many people assume that 1.2 GHz dual core mobile ARM CPU should be almost as fast as a PC CPU running on a similar frequency. They're wrong.

ARM cores are indeed more power efficient per square mm of surface on a same production process than Intel x86 and AMD64 architecture processors. Most of the efficiency comes from a simpler and more space efficient instruction set, but that advantage typically benefits only front-end of the CPU, which is not the biggest spender of those precious miliwatts.

The other reasons why modern dual or quad core mobile phones can run on a fraction of power that notebook or desktop (PC) CPUs need:

less computation units on CPU die (less SIMD, ALU, etc units)
smaller cache than PC CPUs
power gating parts of CPU (but laptop and desktop CPUs also do this for a number of years)
significantly slower DRAM interface than PC CPUs, using slower DDR RAM (LPDDR2)

RAM speed significantly impacts many parts of phone performance. Executing complex JavaScript, image or video processing, Web page rendering are just some of the tasks that significantly benefit from having more RAM bandwidth.

Your ARM device having significantly less of RAM bandwidth is also a big reason why you will probably avoid developing software on your new shiny ASUS Transformer Prime tablet/laptop (though I would certainly try:) )

So how much slower is your Android cell phone RAM than your PC RAM?

Unfortunately, I couldn't find any RAM bench-marking software that would run both on a Linux PC and on a un-rooted android device. There is a nice port of NBench, but NBench is a bigger benchmark and it needs some time before it prints out the one thing we need, the memory index. Also, it doesn't output MB/sec number, which is kind of unfortunate, since it's a really clear metric.

So I found the really simplistic mbw (apt-get install mbw), made it even more simple (removed memcpy tests and left only the dumb array assignment part), and made Android NDK version of it.

RAMbandwidth

https://github.com/vrodic/RAMbandwidth/blob/master/app-debug.apk?raw=true

Source here. Be sure to close any apps before running it on a PC or your phone. Default array size being copied is 20 MB (the app needs 40 MB to perform the test) to better support low memory devices.

Here are some results (20MB array size, 20 repetitions avg, run "mbw -t1 20 -n 20", default settings on RAMbandwidth, on some larger boxes 200MB size was used ):
~12500 MB/sec Intel Core i7-6700, (DDR4 x2 2133 MHz), dedicated GPU
~12300 MB/sec -Intel Core i7-9700 (DDR4 x2 2133 MHz), driving 2560x1440@60Hz display, Ubuntu 19.04, Asrock H310M-STX DeskMini 310
~9000 MB/sec - Intel Core i7-8550U (DDR3 x2 2133 Mhz, Asus UX430UNR)
~9000 MB/sec - Intel Core i7-5600U (DDR3 x2 1600 MHz)
~8200 MB/sec - Asus N56JR (Intel i7-4700HQ, 2x DDR3 1600 Mhz memory)
~6800 MB/sec - Intel Xeon E5-1650 v2 4x DDR3 1600 MHz)
~5400 MB/sec - Intel Xeon X3430, DDR3 memory, under moderate MySQL load( 2009)
~6000 MB/sec - Thinkpad X230 Core i5 3320M (2x DDR3 1600Mhz)

~3800 MB/sec - Core i3-2310M 2x DDR3 1333Mhz

~2200 MB/sec - Intel Core 2 E8200, PC 6400 DDR2 RAM, Desktop PC (2008).

~1100 MB/sec - Intel Core duo L2400, PC 5300 DDR2 RAM on a Thinkpad X60S laptop (2006).

and our mobile contenders

~6000 MB/sec - Xiaomi Pocophone F1 (Snapdragon 845 varies between 5700-7000)
~6000 MB/sec - LG G5 (Snapdragon 820 4 GB LPDDR4 2016, varies between 5800-6500)
~1500 MB/sec - LG G3 (3GB D855 - It varies from 800-1700)
~1200 MB/sec - Raspberry Pi 3
~690 MB/sec - Doogee Valencia2 Y100 Pro
~530 MB/sec- Raspberry Pi 2

~500 MB/sec - Samsung Galaxy S2 (2011)

~250 MB/sec - HTC Desire (2010)
~120 MB/sec - Raspberry PI (2012, under X, fbdev 720p it falls to ~90 MB/sec)

~55 MB/sec - HTC Magic (2009, had to use smaller 10MB array size because of limited RAM available)

Samsung Galaxy S2 sometimes reports around 440 MB/sec, and sometimes 550 MB/sec. I guess it depends where kernel allocates the memory, maybe one of the memory banks shares the bus with the GPU, GSM CPU or some other greedy device.

It should be easy to post some test results of your own hardware, so please share.

EDIT: Check comments for some more results

DORS/CLUC 2019: Mainline kernel on ARM Tegra20 devices that are left behind on 2.6 kernels

May 28, 2019 09:58

Is it possible to take obsolete Android device and port it to mainline kernel with Debian? (make RaspberryPi-like device)

Here is transcript of presentation:

0:00:00.000,0:00:04.529
I hope that you have enjoyed the first day of DORS

0:00:01.949,0:00:08.030
CLUC. My name is Dobrica PavlinuÃÂÃÂ¡iÃÂÃÂ

0:00:04.529,0:00:11.010
and today I will talk about to you about

0:00:08.030,0:00:15.480
how you can take the old Android device

0:00:11.010,0:00:18.840
and port the latest software on it. Over

0:00:15.480,0:00:23.580
there is my prop for this presentation

0:00:18.840,0:00:26.789
which tries to prove that it's really

0:00:23.580,0:00:29.609
possiblei. So the question is: is it

0:00:26.789,0:00:32.430
possible to take the old Android device,

0:00:29.609,0:00:37.020
in this case very old Android device, and

0:00:32.430,0:00:39.420
run latest Linux on it? In my case I

0:00:37.020,0:00:42.600
wanted to make something which would be

0:00:39.420,0:00:45.420
comparable to Respberry Pi which basically

0:00:42.600,0:00:49.980
for me means that I could run Debian on

0:00:45.420,0:00:56.100
it. So what is our device? Our device is

0:00:49.980,0:00:59.280
Tegra tablet from 2011 which is actually

0:00:56.100,0:01:02.280
quite a high-end tablet for that age, it

0:00:59.280,0:01:06.869
has two cores with 1 Gb RAM, it has

0:01:02.280,0:01:10.439
64 Gb of emmc storage, if you

0:01:06.869,0:01:15.140
remember in 2011 that's a huge amount of

0:01:10.439,0:01:20.189
storage, it has quite nice display and

0:01:15.140,0:01:22.740
Wi-Fi, GPS and stuff like that. The things

0:01:20.189,0:01:27.200
which were available from the

0:01:22.740,0:01:29.700
manufacturer is really old 2.6.36 kernel

0:01:27.200,0:01:33.180
fortunately available in source code

0:01:29.700,0:01:38.579
which is very helpful with quite a lot

0:01:33.180,0:01:41.490
of changes, there was also schematic

0:01:38.579,0:01:44.579
available from the OEM manufacturer

0:01:41.490,0:01:49.020
which actually produced that laptop for

0:01:44.579,0:01:51.299
Lenovo which was quite fortunate but not

0:01:49.020,0:01:54.149
as useful as you might think

0:01:51.299,0:01:56.520
so if you can't find the same schematic

0:01:54.149,0:02:01.950
for your Android tablet or device don't

0:01:56.520,0:02:05.729
don't despair it's not obligatory there

0:02:01.950,0:02:08.520
is quite good Tegra support in the

0:02:05.729,0:02:11.250
mainline kernel and as you'll see there

0:02:08.520,0:02:12.950
is also separately developed driver

0:02:11.250,0:02:16.310
which supports

0:02:12.950,0:02:19.819
both 2d and 3d acceleration as you can

0:02:16.310,0:02:22.580
see it's my prop over there, and they

0:02:19.819,0:02:24.830
were available really cheaply locally

0:02:22.580,0:02:28.849
basically I bought it at NjuÃÂÃÂ¡kalo

0:02:24.830,0:02:34.340
which is our local secondhand resale

0:02:28.849,0:02:37.250
kind of site. So what is the first step

0:02:34.340,0:02:39.620
if you want to do something like that? My

0:02:37.250,0:02:41.660
suggestion is to try to find the serial

0:02:39.620,0:02:44.750
port on your device. It would be really

0:02:41.660,0:02:48.500
really helpful when you try to do that

0:02:44.750,0:02:51.290
because first thing you want to do is

0:02:48.500,0:02:57.019
get some kind of feedback from your

0:02:51.290,0:03:00.049
tablet. In my case see the screen didn't

0:02:57.019,0:03:02.000
work so the serial port was really

0:03:00.049,0:03:06.440
invaluable to see whether I am actually

0:03:02.000,0:03:09.140
doing something or not. In this case I

0:03:06.440,0:03:12.950
had a schematic so I knew that there is

0:03:09.140,0:03:16.340
a four pin port somewhere on the tablet

0:03:12.950,0:03:18.680
on which is serial port, I also from the

0:03:16.340,0:03:20.420
schematic knew that this serial port is

0:03:18.680,0:03:23.329
connected directly to the Tegra

0:03:20.420,0:03:26.750
processor which in my case meant that

0:03:23.329,0:03:28.640
this serial port is 1.8 volts which

0:03:26.750,0:03:32.269
means that you don't want to connect it

0:03:28.640,0:03:36.320
to 3.3 volt device although Tegra CPUs

0:03:32.269,0:03:39.140
might be 3.3 volt tolerable but, I don't

0:03:36.320,0:03:44.150
know, you don't want to try that on your

0:03:39.140,0:03:47.239
device. But there is 1.8 volts serial

0:03:44.150,0:03:50.660
cable available from China which are

0:03:47.239,0:03:52.880
basically often often dubbed iPhone

0:03:50.660,0:03:57.430
cables because it uses the same voltage

0:03:52.880,0:03:57.430
but it's just the serial at 1.8 volts.

0:03:58.420,0:04:05.150
Why do I have the picture of the serial

0:04:01.310,0:04:08.090
port? Because in this case this was not

0:04:05.150,0:04:11.630
the only unpopulated connector of the on

0:04:08.090,0:04:14.780
the board with 4 pins, so learn from my

0:04:11.630,0:04:16.579
mistakes and don't try every possible

0:04:14.780,0:04:19.700
connector and until you find the right

0:04:16.579,0:04:21.950
one. It took me quite quite some time to

0:04:19.700,0:04:26.479
figure out that maybe it's under the

0:04:21.950,0:04:30.560
shield and it really was! The other thing

0:04:26.479,0:04:35.570
which you want to have some ability to

0:04:30.560,0:04:38.390
do and you almost always have: basically

0:04:35.570,0:04:42.229
you always have it -- is to be able to run

0:04:38.390,0:04:44.270
your own code on the device. Why do you

0:04:42.229,0:04:47.659
always have that on the Android devices?

0:04:44.270,0:04:50.479
Because when manufacturers create the

0:04:47.659,0:04:53.330
device you should be able

0:04:50.479,0:04:56.539
somehow to load the initial firmware on

0:04:53.330,0:05:00.050
it. So if we have any ARM device either

0:04:56.539,0:05:03.470
Allwinner, Rockchip, Tegra or anything,

0:05:00.050,0:05:07.190
there is a way to actually load your own

0:05:03.470,0:05:09.890
code on it, so that's not the showstopper.

0:05:07.190,0:05:12.860
Tegra is somewhat specific because there

0:05:09.890,0:05:17.320
is ability to lock Tegra bootloader

0:05:12.860,0:05:20.090
so you can't load the non-authorized

0:05:17.320,0:05:22.940
software on it, but this was not the case

0:05:20.090,0:05:25.640
in this case. In Tegra it's called APX

0:05:22.940,0:05:32.870
mode, you can enter it using two keyboard

0:05:25.640,0:05:35.599
presses and with tegrarcm from github

0:05:32.870,0:05:37.760
you can actually create the binary file

0:05:35.599,0:05:40.150
which you can send to your tablet and

0:05:37.760,0:05:43.909
make it do something.

0:05:40.150,0:05:46.219
This ability to to load

0:05:43.909,0:05:46.700
something on the device which is at that

0:05:46.219,0:05:49.430
point

0:05:46.700,0:05:51.530
unmodified enables you to actually

0:05:49.430,0:05:54.169
experiment safely because you can always

0:05:51.530,0:05:56.150
put something, try if it works, and you

0:05:54.169,0:05:59.450
didn't actually change the device itself.

0:05:56.150,0:06:01.520
Although this tablet was so unusable

0:05:59.450,0:06:02.960
because it had so old Anrdoid that

0:06:01.520,0:06:06.919
market didn't support

0:06:02.960,0:06:10.130
it so it wasn't so essential.

0:06:06.919,0:06:11.930
But you know, if this is your only device,

0:06:10.130,0:06:15.500
you probably don't want to break it in

0:06:11.930,0:06:19.070
your first experiment. But just have

0:06:15.500,0:06:20.990
in mind that you won't break your device,

0:06:19.070,0:06:24.530
it's always possible to recover it

0:06:20.990,0:06:27.409
whatever the device is. So far what we

0:06:24.530,0:06:29.479
have? We have working 2.6 kernel with all

0:06:27.409,0:06:32.240
the changes needed to actually make this

0:06:29.479,0:06:35.330
tablet work which isn't really very

0:06:32.240,0:06:36.230
useful but it's nice because we can see

0:06:35.330,0:06:39.380
what

0:06:36.230,0:06:42.440
did they change to make this tablet

0:06:39.380,0:06:45.590
work. I had a serial port and I had the

0:06:42.440,0:06:47.390
ability to run my own code. So what is

0:06:45.590,0:06:49.670
the first step? The first step is to have

0:06:47.390,0:06:52.940
some kind of bootloader which will load

0:06:49.670,0:06:57.230
our Linux kernel. On ARM devices this

0:06:52.940,0:07:00.410
is u-boot and since this tablet is

0:06:57.230,0:07:04.670
actually based on Ventana reference

0:07:00.410,0:07:06.830
design from Nvidia, which you can

0:07:04.670,0:07:11.450
actually figure out by looking at 2.6

0:07:06.830,0:07:14.090
kernel, it was really simple to try to

0:07:11.450,0:07:15.700
compile u-boot, try it out with APX

0:07:14.090,0:07:20.900
and it worked!

0:07:15.700,0:07:22.910
Few steps later I also ... got the

0:07:20.900,0:07:24.890
serial output out of the bootloader

0:07:22.910,0:07:26.840
which was the good first step, but

0:07:24.890,0:07:27.050
display was completely blank. So what did I

0:07:26.840,0:07:29.480
do?

0:07:27.050,0:07:31.910
I took the diff from the 2.6

0:07:29.480,0:07:34.880
kernel and looked what did they modify

0:07:31.910,0:07:38.330
to make the display work? I ported that

0:07:34.880,0:07:44.930
and I got even display in u-boot! mmm

0:07:38.330,0:07:49.940
Victory! I said "port changes" -- it might

0:07:44.930,0:07:52.460
seem extremely complicated or hard. but

0:07:49.940,0:07:54.260
basically that's it! On the left side you

0:07:52.460,0:07:57.200
see the changes which original

0:07:54.260,0:07:59.060
developers made for 2.6 kernel and on

0:07:57.200,0:08:00.950
the right side you can see the changes

0:07:59.060,0:08:03.770
which I made in u-boot to make it work

0:08:00.950,0:08:05.930
basically you compare the names of

0:08:03.770,0:08:12.410
the variables, you change the few ones

0:08:05.930,0:08:14.330
and it works. Once I had u-boot the

0:08:12.410,0:08:16.820
next step was actually to compile the

0:08:14.330,0:08:19.070
kernel and make it work. As I mentioned

0:08:16.820,0:08:21.710
earlier there is the grate driver

0:08:19.070,0:08:26.120
project which basically supports 2d and

0:08:21.710,0:08:28.460
3d acceleration on Tegra devices so I

0:08:26.120,0:08:33.530
actually started with it because I

0:08:28.460,0:08:36.700
wanted 2d and 3d and video decoding I

0:08:33.530,0:08:43.010
started in the same way I looked at 2.6

0:08:36.700,0:08:46.340
kernel tried to port the display, it

0:08:43.010,0:08:49.520
should be possible to define display

0:08:46.340,0:08:53.120
in device tree itself, for some reason

0:08:49.520,0:08:56.990
it did not work for me so this was few lines

0:08:53.120,0:09:00.410
of diff in kernel itself, but other

0:08:56.990,0:09:02.900
than that, all the other things were

0:09:00.410,0:09:06.260
basically device tree configuration.

0:09:02.900,0:09:09.770
I had to configure the buttons

0:09:06.260,0:09:14.510
on the laptop to generate keyboard

0:09:09.770,0:09:16.700
events and from 2.6 kernel it wasn't

0:09:14.510,0:09:19.160
clear whether the button is both pull-up

0:09:16.700,0:09:22.010
or pulldown but you try one, you try the

0:09:19.160,0:09:24.320
other, and if you make a mistake, if you

0:09:22.010,0:09:26.660
said that button is pulled down and it's

0:09:24.320,0:09:28.190
pull up the thing which will happen is

0:09:26.660,0:09:31.580
that when you press the button it

0:09:28.190,0:09:34.100
it won't release, so you will figure it

0:09:31.580,0:09:35.750
out. Basically some buttons are pull up

0:09:34.100,0:09:39.130
some buttons are pull down, you just

0:09:35.750,0:09:42.650
experiment a little and it will work out.

0:09:39.130,0:09:44.720
Then I added a few additional

0:09:42.650,0:09:47.630
modules which were supported in upstream

0:09:44.720,0:09:50.930
kernel already, like a temperature sensor

0:09:47.630,0:09:54.800
compass and there is also in tablet

0:09:50.930,0:09:56.690
accelerometer which should be

0:09:54.800,0:09:59.300
supported by the kernel module, but

0:09:56.690,0:10:03.470
currently doesn't work -- work in

0:09:59.300,0:10:06.050
progress -- but all in all this diff --stat

0:10:03.470,0:10:09.830
at the bottom of the slide are all the

0:10:06.050,0:10:13.040
changes which were required to make this

0:10:09.830,0:10:19.270
tablet, over there, which actually started

0:10:13.040,0:10:24.800
screensaver, working on the latest kernel

0:10:19.270,0:10:27.170
not really so hard at all, right? As the

0:10:24.800,0:10:29.540
next step and probably the most

0:10:27.170,0:10:32.030
important thing which I learned during

0:10:29.540,0:10:35.300
this process, is actually that you want

0:10:32.030,0:10:37.190
to develop using NFS root. You don't

0:10:35.300,0:10:40.040
actually want to experiment on the

0:10:37.190,0:10:45.020
device itself because it's so convenient

0:10:40.040,0:10:47.030
to actually edit files in VI on your NFS

0:10:45.020,0:10:50.720
server which in my case is just a

0:10:47.030,0:10:52.550
ordinary laptop instead of editing it on

0:10:50.720,0:10:54.230
the device itself especially if the

0:10:52.550,0:10:57.520
device itself doesn't have the keyboard.

0:10:54.230,0:11:00.500
Right now I do have the keyboard but

0:10:57.520,0:11:03.620
when I started I didn't have any device

0:11:00.500,0:11:06.650
and it's you know it's really much more

0:11:03.620,0:11:10.940
variable to do that on on your normal

0:11:06.650,0:11:15.950
development machine. NFS will also enable

0:11:10.940,0:11:18.140
you to to try different devices but the

0:11:15.950,0:11:21.620
prerequisite for that is actually to

0:11:18.140,0:11:23.290
have the USB Ethernet device which is

0:11:21.620,0:11:26.320
supported by u-boot

0:11:23.290,0:11:29.360
unfortunately u-boot support very few

0:11:26.320,0:11:33.250
USB Ethernet dongle so you will have to

0:11:29.360,0:11:36.529
find the one which is supported or port

0:11:33.250,0:11:39.350
changes from some other USB dongle which

0:11:36.529,0:11:42.650
is also not that hard but it wasn't

0:11:39.350,0:11:47.210
needed because I actually had a dongle

0:11:42.650,0:11:49.100
which is supported. The second

0:11:47.210,0:11:53.330
interesting thing I learned here is the

0:11:49.100,0:11:57.160
one marked here in yellow which is that

0:11:53.330,0:12:00.950
the kernel configuration

0:11:57.160,0:12:04.460
because you say to u-boot ok please

0:12:00.950,0:12:06.860
use the DHCP acquire MAC address and

0:12:04.460,0:12:09.830
then load the kernel and initramfs

0:12:06.860,0:12:13.160
from the from the server and once

0:12:09.830,0:12:16.880
you start the kernel the kernel also has

0:12:13.160,0:12:20.900
the option to acquire address over DHCP but

0:12:16.880,0:12:24.920
unfortunately that didn't work I suspect

0:12:20.900,0:12:28.520
that it's problem with initialization of

0:12:24.920,0:12:31.150
the USB interface in kernel so the

0:12:28.520,0:12:34.430
interface is not initialized correctly

0:12:31.150,0:12:37.490
or something, but you can always hard

0:12:34.430,0:12:40.880
code the IP address and that worked. If

0:12:37.490,0:12:42.709
you want more info about making u-boot

0:12:40.880,0:12:45.770
work with the NFS root and

0:12:42.709,0:12:49.010
configuration of dnsmasq the last link

0:12:45.770,0:12:52.910
here is actually the wiki page in which

0:12:49.010,0:12:55.220
you can find more information. And then I

0:12:52.910,0:12:57.200
had the tablet which was somewhat

0:12:55.220,0:13:00.860
working but the problem was that I

0:12:57.200,0:13:03.560
couldn't charge it. Since this tablet is

0:13:00.860,0:13:06.830
from 2011 you would expect that the

0:13:03.560,0:13:09.950
battery is quite dead and it really is

0:13:06.830,0:13:12.260
quite dead but it's really annoying that

0:13:09.950,0:13:13.850
you can actually work several hours on

0:13:12.260,0:13:16.130
your tablet and then you have to take

0:13:13.850,0:13:16.400
another device which was charging during

0:13:16.130,0:13:20.600
that

0:13:16.400,0:13:25.430
time so I wanted to somehow make it work

0:13:20.600,0:13:27.590
to make it work always, to have it

0:13:25.430,0:13:32.560
always powered on and to be able to charge

0:13:27.590,0:13:35.510
it from the USB. The problem is that this

0:13:32.560,0:13:38.540
particular tablet is very sensitive to

0:13:35.510,0:13:42.580
the 5 volt rail and if you don't have

0:13:38.540,0:13:45.920
stable 5 volt rail it will try to

0:13:42.580,0:13:48.530
to pull as much as 2 amps if the battery

0:13:45.920,0:13:50.870
is totally flat and if the voltage drops

0:13:48.530,0:13:52.910
a little bit below 5 volts it will

0:13:50.870,0:13:55.580
just give up and say okay I won't charge

0:13:52.910,0:13:58.010
so the tablet was charging quite nice

0:13:55.580,0:14:00.800
when powered off but didn't charge with

0:13:58.010,0:14:03.590
power on, so what could I do?

0:14:00.800,0:14:08.660
Other than draw nice graphs which show

0:14:03.590,0:14:11.930
my problems? Well I can look at 2.6

0:14:08.660,0:14:14.630
kernel and see what did they do to actually

0:14:11.930,0:14:19.130
make it work? This tablet is also

0:14:14.630,0:14:21.650
somewhat specific in regards to other

0:14:19.130,0:14:25.970
Android tablets because it has another

0:14:21.650,0:14:28.250
processor which is 8051 core which

0:14:25.970,0:14:31.100
basically talks with battery so I don't

0:14:28.250,0:14:33.950
have direct connection with the

0:14:31.100,0:14:36.860
battery controller but I have it through

0:14:33.950,0:14:39.620
the firmware in that microcontroller

0:14:36.860,0:14:43.610
which is connected to the Tegra device

0:14:39.620,0:14:46.400
using i2c. This was in one sense annoying

0:14:43.610,0:14:47.860
because if I could directly drive the

0:14:46.400,0:14:50.480
battery charger it would be much easier

0:14:47.860,0:14:53.000
but on the other hand that meant that

0:14:50.480,0:14:55.850
the solution was rather simple I just

0:14:53.000,0:14:59.060
had to send one i2c

0:14:55.850,0:15:02.080
command copied from the 2.6 kernel and

0:14:59.060,0:15:07.850
the tablet would start charging

0:15:02.080,0:15:12.560
win/win/win and as you can see on the

0:15:07.850,0:15:17.240
demo after recompiling the whole GL

0:15:12.560,0:15:21.230
stack including libdrm, mesa and opentegra

0:15:17.240,0:15:26.900
video driver I actually have x11 running

0:15:21.230,0:15:29.680
on it without any problems whatsoever

0:15:26.900,0:15:32.200
So what works and what doesn't?

0:15:29.680,0:15:34.000
from the i2c devices on the left which

0:15:32.200,0:15:36.820
are basically the list of the devices

0:15:34.000,0:15:40.240
from the 2.6 kernel we can see that

0:15:36.820,0:15:43.510
audio, charging, compas, power and

0:15:40.240,0:15:45.910
temperature are working as is, the things

0:15:43.510,0:15:47.620
which are denoted by the small hand are

0:15:45.910,0:15:49.500
actually the things which I had to do

0:15:47.620,0:15:53.140
something

0:15:49.500,0:15:57.370
unfortunately the cameras are not

0:15:53.140,0:16:00.820
supported but you know they're lousy

0:15:57.370,0:16:03.490
cameras from 2011 and this tablet is

0:16:00.820,0:16:07.149
still better than the Raspberry Pi

0:16:03.490,0:16:11.709
diplay works, HDMI probably works, I didn't

0:16:07.149,0:16:14.680
really test it the main drawback is that

0:16:11.709,0:16:17.740
the touchscreen on this device is

0:16:14.680,0:16:20.970
the SPI device which currently doesn't

0:16:17.740,0:16:24.490
work for me the SPI doesn't work at all

0:16:20.970,0:16:26.680
keys were really easy those were the key

0:16:24.490,0:16:31.450
is connected to GPIO just a little bit

0:16:26.680,0:16:35.290
of device tree, the vibrator there is a

0:16:31.450,0:16:38.230
small vibrating motor in the tablet,

0:16:35.290,0:16:40.209
actually doesn't work for me I really

0:16:38.230,0:16:44.860
don't know why there is nothing special

0:16:40.209,0:16:47.680
in 2.6 kernel for it but if I toggle the

0:16:44.860,0:16:51.520
pin nothing happens and it does work in

0:16:47.680,0:16:54.339
2.6 so more work needs to be done

0:16:51.520,0:16:58.680
there is also the proximity sensor which

0:16:54.339,0:17:02.500
works also one simple GPIO there is the

0:16:58.680,0:17:05.920
Wi-Fi and 3G modem which works because

0:17:02.500,0:17:08.589
it's the simple USB device there is the

0:17:05.920,0:17:11.679
internal flash connected to MMC which

0:17:08.589,0:17:14.230
also works and the SD card I think that

0:17:11.679,0:17:16.660
SD card actually works but it's a big SD

0:17:14.230,0:17:24.250
card so I just didn't have any handy to

0:17:16.660,0:17:28.390
test it. [Adapter?] yeah sure but with 64 Gb

0:17:24.250,0:17:31.000
of emmc which has 40 Mb of

0:17:28.390,0:17:35.890
transfer rate why would I even try the

0:17:31.000,0:17:39.100
SD card right? So was it worth it? For me

0:17:35.890,0:17:42.320
it surely is! If the goal was to

0:17:39.100,0:17:48.440
be able to type apt-get update

0:17:42.320,0:17:50.750
I have achieved that goal. So if you have

0:17:48.440,0:17:53.960
more devices you can spend one of

0:17:50.750,0:17:55.850
them to actually figure out what is what

0:17:53.960,0:17:59.389
is on the board, and this is one of the

0:17:55.850,0:18:01.850
tablets disassembled into into separate

0:17:59.389,0:18:07.899
pieces still working as you can see it

0:18:01.850,0:18:11.480
has the LED it works but there are also

0:18:07.899,0:18:14.480
few things left to do. For a start the

0:18:11.480,0:18:17.120
SPI controller doesn't work which is

0:18:14.480,0:18:18.860
quite strange I think I configured

0:18:17.120,0:18:21.559
everything but surely there is the

0:18:18.860,0:18:29.059
problem between me and and the code I

0:18:21.559,0:18:31.940
wrote. In mainstream kernel there is similar

0:18:29.059,0:18:36.799
driver for the touchpad which is used in

0:18:31.940,0:18:40.730
surface Microsoft Surface 3 but that

0:18:36.799,0:18:43.519
driver actually use ACPI tables to

0:18:40.730,0:18:45.259
initialize and on the arm devices we

0:18:43.519,0:18:47.299
would need the device tree to do that

0:18:45.259,0:18:49.659
and I actually wrote the code which

0:18:47.299,0:18:51.649
actually query the device tree

0:18:49.659,0:18:54.440
it's really simple you just

0:18:51.649,0:18:56.330
you just add a few defines in kernel

0:18:54.440,0:18:59.269
module and it will also query the device

0:18:56.330,0:19:03.769
tree but since the SPI doesn't work for

0:18:59.269,0:19:07.100
me currently unfortunately touch as

0:19:03.769,0:19:10.039
of today doesn't still work embedded

0:19:07.100,0:19:12.769
controller will need a little bit more

0:19:10.039,0:19:15.139
work and it's actually more essential

0:19:12.769,0:19:18.320
because I would really like to be able

0:19:15.139,0:19:21.019
to plug in the power and for

0:19:18.320,0:19:22.899
battery to start charging immediately as

0:19:21.019,0:19:26.330
opposed to me starting the shell script

0:19:22.899,0:19:30.919
but you know for now it actually works

0:19:26.330,0:19:33.169
and cameras are are supported the

0:19:30.919,0:19:36.470
problem with cameras it should be really

0:19:33.169,0:19:39.610
easy from the perspective of the kernel

0:19:36.470,0:19:42.889
driver developer the cameras are

0:19:39.610,0:19:45.080
relatively simple i2c devices because

0:19:42.889,0:19:48.320
you just have to set up the camera

0:19:45.080,0:19:51.470
the cameras are CSI and they will start

0:19:48.320,0:19:54.400
streaming frames to memory of Tegra

0:19:51.470,0:19:56.559
Tegra hardware support decode of

0:19:54.400,0:20:03.730
that in memory and all should be golden

0:19:56.559,0:20:06.550
but the video4linux 2 API in kernel

0:20:03.730,0:20:09.040
is currently changing so all the

0:20:06.550,0:20:11.590
examples for the camera similar to

0:20:09.040,0:20:14.260
mine are actually examples for the old

0:20:11.590,0:20:19.990
way of doing things as opposed to new

0:20:14.260,0:20:23.640
one so this is somewhat something which

0:20:19.990,0:20:27.340
I have to do at some later date

0:20:23.640,0:20:29.410
still real Linux distribution on

0:20:27.340,0:20:32.410
this device is much more useful than

0:20:29.410,0:20:35.140
obsolete Android and if you enjoy this

0:20:32.410,0:20:38.260
or you have some Tegra 2 device you can

0:20:35.140,0:20:40.870
find additional notes here on the other

0:20:38.260,0:20:43.630
hand if you don't have Tegra device but

0:20:40.870,0:20:46.840
some other Android tablet on which you

0:20:43.630,0:20:49.660
want to do something like this you can

0:20:46.840,0:20:51.400
try some of the following links if you

0:20:49.660,0:20:55.300
have allwinner

0:20:51.400,0:20:57.280
or rockchip device I suggest to take a

0:20:55.300,0:21:00.850
look at armbian which is probably

0:20:57.280,0:21:04.540
the most well-known and best ARM based

0:21:00.850,0:21:06.760
Linux distro for the devices if you have

0:21:04.540,0:21:12.400
the OMAP based device which is

0:21:06.760,0:21:16.540
basically the Nexus 7 or older Nexus

0:21:12.400,0:21:19.000
phones there is talk from fosdem which

0:21:16.540,0:21:21.490
goes into more details what you can do

0:21:19.000,0:21:26.380
they are also quite well supported in -

0:21:21.490,0:21:29.350
in mainstream kernel and the last

0:21:26.380,0:21:33.429
alternative is postmarketOS which goal

0:21:29.350,0:21:36.280
is to bring longer life to all the

0:21:33.429,0:21:39.160
devices so in a sense similar goal to

0:21:36.280,0:21:43.600
mine it's based on Alpine which is

0:21:39.160,0:21:46.090
basically why I didn't use that but it

0:21:43.600,0:21:48.059
does have the support for Samsung Galaxy

0:21:46.090,0:21:53.290
Tab 10 which is also

0:21:48.059,0:21:55.840
tera device and this source

0:21:53.290,0:21:59.110
code for this device actually got me

0:21:55.840,0:22:01.179
the courage to actually try this because

0:21:59.110,0:22:03.820
I could see all the changes between main

0:22:01.179,0:22:05.980
line and support needed for one Tegra

0:22:03.820,0:22:07.540
device which wasn't supported before and

0:22:05.980,0:22:10.960
I said

0:22:07.540,0:22:13.500
this doesn't seem so hard it wasn't it

0:22:10.960,0:22:18.490
wasn't useful for any practical

0:22:13.500,0:22:20.500
practical I didn't copy any code I the

0:22:18.490,0:22:23.740
tablets are different enough that it

0:22:20.500,0:22:26.260
wasn't directly reusable but it gave me

0:22:23.740,0:22:29.380
the courage to actually

0:22:26.260,0:22:32.830
try it out so hopefully this will

0:22:29.380,0:22:35.770
motivate you to revive some of your old

0:22:32.830,0:22:44.320
Android devices. Do you have any

0:22:35.770,0:22:47.890
questions? this the same grate driver

0:22:44.320,0:22:50.890
supports Tegra 3 the Tegra 2 and newer

0:22:47.890,0:22:53.320
so Tegra 3 is also supported

0:22:50.890,0:22:56.080
although depending on which Tegra you

0:22:53.320,0:23:00.030
have these days they usually have locked

0:22:56.080,0:23:00.030
bootloader but...

0:23:00.630,0:23:10.320
if you can update Android on your

0:23:06.690,0:23:15.510
Tegra device there is a possibility to

0:23:10.320,0:23:18.900
actually replace the Android kernel with

0:23:15.510,0:23:22.470
the kernel which has kexec as opposed

0:23:18.900,0:23:25.650
of using u-boot to boot the kernel you

0:23:22.470,0:23:27.390
actually install your own kernel which

0:23:25.650,0:23:29.910
is some version which is supported on

0:23:27.390,0:23:32.400
your device and you then you can do with

0:23:29.910,0:23:35.040
the kexec to the current kernel so

0:23:32.400,0:23:38.570
it's also possible I actually do have I

0:23:35.040,0:23:41.130
actually got a new friend from Germany

0:23:38.570,0:23:42.960
during this project because I started

0:23:41.130,0:23:45.180
documenting everything on the wiki as I

0:23:42.960,0:23:48.330
was working and he contacted me and said

0:23:45.180,0:23:51.330
oh I have the Tegra tablet also for two

0:23:48.330,0:23:53.130
years I'm so happy I found you and he

0:23:51.330,0:23:55.140
actually sent me the keyboard this is

0:23:53.130,0:23:58.110
why I now have the keyboard and didn't

0:23:55.140,0:24:03.600
have it and this one is locked so I will

0:23:58.110,0:24:05.430
try that that kexec trick in the

0:24:03.600,0:24:08.190
future and document it on the wiki so

0:24:05.430,0:24:20.580
this might be helpful any other

0:24:08.190,0:24:24.510
questions? [How much did it cost?] It was it was between 100

0:24:20.580,0:24:27.060
between 80 and 100 kunas depending on

0:24:24.510,0:24:30.470
the on the state of disrepair which is

0:24:27.060,0:24:35.760
for the international audience between

0:24:30.470,0:24:39.770
11-12 euros and like 14 right so they

0:24:35.760,0:24:39.770
were really cheap I have a bunch of them

0:24:43.290,0:24:50.660
[Applause]

Power cycle network switch using Arduino and relay

January 22, 2019 08:31

Our top-of-switch rack decides to die randomly from time to time. It was somewhat inconvenient since it also killed most of our infrastructure including primary and secondary DNS so I needed a solution quickly. Since different rack is still on the network, I should be able to hack something and finally connect my Arduino knowledge and sysadmin realm, right? Think of it as power cycle watchdog based on network state.

First thing was to figure out what was happening with the switch. It seemed like it was still working (LEDs did blink), but only thing that helped was power cycle. So as a first strep, I connected serial console (using RS-232 extension cable) to on-board serial port (since it doesn't seem to work using cheap CH340 based USB serial dongles) and I didn't expect this:

0x37491a0 (bcmCNTR.0): memPartAlloc: block too big 6184 bytes (0x10 aligned) in partition 0x30f6d88
0x3cd6bd0 (bcmCNTR.1): memPartAlloc: block too big 6184 bytes (0x10 aligned) in partition 0x30f6d88
0x6024c00 (simPts_task): memPartAlloc: block too big 1576 bytes (0x10 aligned) in partition 0x30f6d88
0x6024c00 (simPts_task): memPartAlloc: block too big 1576 bytes (0x10 aligned) in partition 0x30f6d88

When I google messages like this I get two types of answers:

beginner questions about VxWorks which summ up to: you have memory leak
errors from switches with boardcomm chipset from various vendors

There is basically no solution. We are running latest firmware, and internet doesn't have any idea what to do.
Serial console did emit a lot of messages, but didn't respond to input at all. I would at last expect that watchdog timer in the switch will reset it once it manages to fragment it's own memory so much that it has stopped forwarding packets, oh well.... What else can I do?

IEC power cable with relay

I wanted something what I can plug in between the existing switch with IEC power connector with USB on the other end that can be plugged into any USB port for control.

Since this is 220V project (and my first important one), I tried to do it as safe as possible.

I started with a power cable, that I cut in half and put ferrules on all wires to be sure that connectors will grip those wires well.
Then I replaced the power plug with IEC connector so it's can be inserted in any power cable. In this case, we soldered wires ends, since ferrules where too big to fit into connector housing. We did wrap a wire around a screw in a connector correctly, so tightening the screw will not displace the wire.
Finally I connected cheap 10A 250VAC relay which should be enough for fully loaded 48 port gigabit network switch that draws round 80W.
To make sure that rest of the system can't just power cycle device connected at any time, I connected live wire through normally closed pins on a relay. This means that this cable should work as-is (without powering it at all) and when powered, since board has pull-up resistor on the relay to VCC, the relay will be in the same sate, passing power to device.
Finally I checked all three power cable wires with multi-meter and got around 0.2 ohms which mans that whole thing works for now.

At this point we should note that this relay board has only three pins (IN, GND and VCC) and has no optical isolation to 220V side. Since isolation would require us to provide additional power supply for 220V side, it was acceptable a risk.

Putting it into a box

I really wanted to somehow fix wires and protect the bottom of the relay board (which has 220V on it) from shorting to something, so I used an old box from a dairy product and created a housing for electronics.

If you look carefully, you will notice that I had to cut the case all the way through to pass through the power cable (that has a zip-tie on inside to prevent it from pulling out). The Case will be fixed using hot glue and a lid, so this won't be a problem.
Warning and label on the lid is also nice touch, and shouldn't be skipped when creating a thing which you won't be only user of.

Arduino software

You will also notice that relay is connected to A7, which didn't work out. Let me explain:
The idea is to use Arduino default pin state (INPUT) as a state in which the pin will stay most of the time. This makes pin floating, and we can inspect pull-up on relay board and report if we see it. When we want to activate the relay, we'll flip pin to output, pull it down, and activate the relay.
Code is available at https://github.com/dpavlin/Arduino-projects/blob/nuc/power_cycle/power_cycle.ino and it can't be much simpler:

/*
 * power cycle switch
 * 
 * relay is connected across 5V relay through normally closed pins so that failure of arduino doesn't kill power to switch.
 * to activate relay on this board, signal pin has to be pulled to ground.and coil draw is 76 mA when active
 * board has pull up on input pin to it's vcc
*/

#define RELAY_PIN 2

void setup() {
  Serial.begin(115200);

  pinMode(LED_BUILTIN, OUTPUT);

  pinMode(RELAY_PIN, INPUT); // don't modify pin state
  Serial.print("Relay pin on reset: ");
  Serial.println(digitalRead(RELAY_PIN));
}

void loop() {
  if ( Serial.available() ) {
    char c = Serial.read();
    if ( c == '0' ) {
      Serial.print("L");
      pinMode(RELAY_PIN, OUTPUT);
      digitalWrite(RELAY_PIN, LOW); // activate relay

      digitalWrite(LED_BUILTIN, HIGH); // led on
    } else if ( c == '1' ) {
      Serial.print("H");
      pinMode(RELAY_PIN, INPUT);

      digitalWrite(LED_BUILTIN, LOW); // led off
    } else {
      Serial.print(c);
    }
  }
}

Simple is good: I toyed with idea of automatically releasing the relay from Arduino code, and when I started to implement timeout configuration on Arduino side, I remembered what this will be plugged into random server USB port, without avrdude and any handy way to update firmware on it, so I decided to just leave simplest possible commands:

1 - ON (outputs H) - power on, default
0 - OFF (outputs L) - power off, relay active

Hot glue galore

Then, I applied liberal amount of hot-glue to fix power cables and board in place. It worked out pretty well. You will also notice that the relay pin has moved to D2.

Installation

And here it is, installed between existing switch power cable and switch, connected to only USB port still available in rack which is still on network.

cron and serial port

Idea is simple: we'll use cron to ping primary and secondary DNS IP addresses and if any of these fail, we'll send 0 to turn power off, wait 3 seconds, and send 1 to turn power back on.
Implementation, however, is full of quirks, mostly because we don't want to depend on additional utilities installed, and we need to wait for Arduino to reset after connecting to serial port (and to give it time to display value of relay pin) before we start turning power off.

#!/bin/sh -e

ping -q -c 5 193.198.212.8 > /dev/shm/ping && ping -q -c 5 193.198.213.8 >> /dev/shm/ping || (

test -e /dev/shm/reset && exit 0 # reset just once
cp /dev/shm/ping /dev/shm/reset  # store failed ping

date +%Y-%m-%dT%H:%M:%S
cat /dev/shm/ping

dev=/dev/ttyUSB0

trap "exit" INT TERM
trap "kill 0" EXIT

stty -F $dev speed 115200 raw
cat < $dev &
(
        echo
        sleep 3 # wait for reset and startup message
        echo 0  # off
        sleep 3
        echo 1  # on
        sleep 1
) | cat > $dev

kill $!

) # ping subshell

It's started from crontab with user which has dialout group membership so he can open /dev/ttyUSB0:

dpavlin@ceph04:~$ ls -al /dev/ttyUSB0 
crw-rw---- 1 root dialout 188, 0 Dec 28 01:44 /dev/ttyUSB0
dpavlin@ceph04:~$ id
uid=1001(dpavlin) gid=1001(dpavlin) groups=1001(dpavlin),20(dialout),27(sudo)
dpavlin@ceph04:~$ crontab -l | tail -1
*/1 *  *   *   *     /home/dpavlin/sw-lib-srv-power-cycle.sh

This will execute script every minute. This allows us to detect error within minute. However, switch boot takes 50s, so we can't just run this script every minute, because it will result in constant switch power cycles. But since we are resetting switch just once this is not a problem.

With this in place, your network switch will not force you to walk to it so you can power cycle it any more. :-)
And it's interesting combination of sysadmin skills and electronics which might be helpful to someone.

remote access

If we want to access our servers while switch doesn't work, it's always useful to create few shell scripts on remote nodes which will capture IP addresses and commands which you will need to execute to recover your network.

dpavlin@data:~/ffzg$ cat ceph04-switch-relay.sh 
#!/bin/sh -xe

ssh 193.198.212.46 microcom -p /dev/ttyUSB0 -s 115200

dpavlin@data:~/ffzg$ cat r1u32-sw-rack3.sh 

#!/bin/sh

ssh 193.198.212.43 microcom -p /dev/ttyS1 -s 9600

LSI MegaRAID slow compared to md

September 27, 2018 12:48

This week I learned valuable lesson: if you are using MIPS from 2008 in your RAID controller, you can't really expect it to be faster than more modern Intel CPU when doing RAID 10 on disks.

It all started with failure of SSD in our bcache setup which sits on top of MegaRAID RAID10 array. Since this required me take one of ganeti nodes down, it was also a good opportunity to add one more disk (we where running 6 disks and one SSD) and switch to software md RAID10 so we can use all 7 disks in RAID10. In this process, I did some benchmarking and was shocked with results.

First, let's see original MegaRAID configuration:

# lsblk --scsi -m
NAME HCTL       TYPE VENDOR   MODEL             REV TRAN NAME   SIZE OWNER GROUP MODE
sda  0:0:6:0    disk ATA      WDC WD1002FBYS-1 0C12      sda  931.5G root  disk  brw-rw----
sdb  0:0:7:0    disk ATA      INTEL SSDSC2BW24 DC32      sdb  223.6G root  disk  brw-rw----
sdc  0:2:0:0    disk DELL     PERC H310        2.12      sdc    2.7T root  disk  brw-rw----
# hdparm -tT /dev/sdc

/dev/sdc:
 Timing cached reads:   13920 MB in  1.99 seconds = 6981.76 MB/sec
 Timing buffered disk reads: 1356 MB in  3.00 seconds = 451.81 MB/sec

and let's compare that with JBOD disks on controller and creating md array:

# hdparm -Tt /dev/md0
/dev/md0:
 Timing cached reads:   13826 MB in  1.99 seconds = 6935.19 MB/sec
 Timing buffered disk reads: 1888 MB in  3.01 seconds = 628.05 MB/sec

So, TL;DR is that you are throwing away disk performance if you are using hardware RAID. You didn't expect that? I didn't. In retrospect it's logical that newish Intel CPU can process data much faster than slowish MIPS on RAID controller, but on the other hand only difference is RAID overhead because same controller is still handling disks with software raid.

I also wrote document with a lot of console output and commands to type if you want to do the same: https://github.com/ffzg/gnt-info/blob/master/doc/megaraid-to-md.txt

Edit any textarea in vi with syntax highlight

July 11, 2018 18:45

Today I was explaining xclip utility and found this useful snippet which I wrote back in 2011 that allows you to edit browser textarea in terminal using vi with syntax highlighting.

#!/bin/sh

# workflow:
# 1. focus textarea in browser you want to edit
# 2. press ctrl+a then ctrl+c
# 3. switch to terminal and start this script with optional extensioni for highlight: xclip-vi html
# 4. edit file in vi, and save it
# 5. switch back to browser, and press ctrl+v in already selected textarea

ext=$1
test -z "$ext" && ext=txt

xclip -out > /tmp/$$.$ext && vi /tmp/$$.$ext && xclip -in -selection clipboard < /tmp/$$.$ext

xclip-vi.sh github gist

Acer Nitro 5 with nVidia 1050 on Linux (Ubuntu 18.04)

July 02, 2018 15:38

Acer Nitro 5 with nVidia 1050 GPU is an interesting beast.

- HDMI Output is wired to the NVIDIA chip

- Internal display wired to Intel GPU.

This is different than Optimus where both outputs are driven by the integrated GPU, and is actually more efficient since it doesn't spend system RAM bandwidth for display refresh or for copying the GPU discrete GPU frame-buffer to the internal GPU frame-buffer when rendering using the.

So it's important that switching between using external monitor and internal laptop panel is handled gracefully.

Windows driver probably handles this automatically (though it's possible it has to be a special Acer build). It maybe even allows both GPUs to be active at the same time (Intel for internal, nVidia for external display)

But Linux nVidia driver typically works either in Optimus mode where both internal and external device is driven by Intel GPU or in traditional mode where monitor is only connected to the discrete GPU. Bumblebee project supports a more flexible setup, but is maybe a bit more difficult to configure

This is a short guide to enable switching between the either using external monitor with nVidia or Internal monitor with Intel GPU somewhat easily on Ubuntu 18.04:

First install nvidia proprietary driver :
- apt install nvidia-driver-390 nvidia-prime
- edit /etc/default/grub so that GRUB_CMDLINE_LINUX_DEFAULT line has nomodeset option eg, GRUB_CMDLINE_LINUX_DEFAULT="quiet splash nomodeset" & run sudo update-grub
- restart machine while external HDMI monitor is connected

To switch to internal monitor:
- run sudo prime-select intel while on external monitor (or if you don't have the external monitor, Ctrl-Alt-F3 to text console and run sudo prime-select intel)
- reboot
- to enable acceleration for intel remove nomodeset from the kernel command line

To switch back to external monitor:
- run sudo prime-select nvidia
- reboot
- make sure there is nomodeset in the kernel command line

Making a shell script that does nomodeset grub switching and prime-select in a single step should be possible, and a situation where there is no nomodeset but nvidia is configured (GDM is stuck in neverending start loop) can be fixed by manually adding nomodeset in the grub menu

If you want to try a more flexible setup with Bumblebee, try these links

https://unix.stackexchange.com/questions/321151/do-not-manage-to-activate-hdmi-on-a-laptop-that-has-optimus-bumblebee

https://github.com/Bumblebee-Project/Bumblebee/wiki/Multi-monitor-setup

https://wiki.archlinux.org/index.php/bumblebee#Output_wired_to_the_NVIDIA_chip

Running legacy binary junk after upgrade in chroot

June 13, 2018 15:47

These days, my work seems more like archeology than implementing newest and coolest web technologies, but someone has to keep an eye on old web servers which are useful to people.

In this case, it's two major Debian releases upgrade which of course did result in some amount of breakage as you would expect. Most of breakage was easy to fix, but we had one site which had only pyc files, and python2.6 with correct modules wasn't supported on current distribution.

First idea was to use python de-compiler to generate source python files, and this only got me to different python errors which I didn't know how to fix. so, this was dead end.

I did have a backup of machine before upgrade on zfs pool, so next logical idea was to run minimal amount of old binaries to keep the site up. And I decided to do it in chroot so I can easily share mysql socket from current installation into pre-upgrade squeeze. Let's see what was involved in making this happen.

1. make writable copy of filesystem

Our backups are on zfs pool, so we cloned all three disks into same layout as they are mounted on filesystem:

2018-06-08.10:07:53 zfs clone lib15/backup/pauk.ffzg.hr/0@2018-05-01 lib15/export/pauk.ffzg.hr
2018-06-08.10:08:19 zfs clone lib15/backup/pauk.ffzg.hr/1@2018-05-01 lib15/export/pauk.ffzg.hr/home
2018-06-08.10:08:38 zfs clone lib15/backup/pauk.ffzg.hr/2@2018-05-01 lib15/export/pauk.ffzg.hr/srv

2. nfs export it to upgraded server

Take a note of crossmnt parameter, this will allow us to mount all descending filesystems automatically.

root@lib15:~# grep pauk /etc/exports 
/lib15/export/pauk.ffzg.hr 10.21.0.2(rw,fsid=2,no_subtree_check,no_root_squash,crossmnt)
root@lib15:~# exportfs -va
exporting 10.21.0.2:/lib15/export/pauk.ffzg.hr

3. mount nfs export

root@pauk:~# grep squeeze /etc/fstab
10.21.0.215:/lib15/export/pauk.ffzg.hr  /mnt/squeeze    nfs     defaults,rw     0 0

4. start chroot with old version of service

To make this work, I first edited apache configuration in chroot to start it on different port, and configured front-end server to redirect site to new port.

root@pauk:/home/dpavlin# cat chroot-squeeze.sh 

to=/mnt/squeeze

mount --bind /dev $to/dev
mount --bind /sys $to/sys
mount --bind /proc $to/proc

mount --bind /var/run/mysqld/ /mnt/squeeze/var/run/mysqld/

/usr/sbin/chroot $to /etc/init.d/apache2 start

5. redirect apache traffic to server in chroot

You will have to insert something like this in current apache configuration for vhost:

<Proxy http://127.0.0.1:18080/*>
    Order allow,deny
    Allow from all
</Proxy>
<Location /german2/>
    RequestHeader set Host "www.ffzg.unizg.hr"
    ProxyPreserveHost On
    ProxyPass        http://127.0.0.1:18080/german/
    ProxyPassReverse http://127.0.0.1:18080/german/
</Location>

You might scream at me that in days of containers, systemd and other better solutions chroot is obsolete. In my opinion, it's just a right tool for a job from time to time, if you have backup :-)

DORS/CLUC 2017: bro - what's in your network?

June 10, 2018 10:35

This year on DORS/CLUC 2017, I talked about bro which is great network analysys tools everybody should know, but dispite it's long history is not as well known as you would suspect.

DORS/CLUC 2018: linux+sensor+device-tree+shell=IoT ?

June 10, 2018 10:18

This year on DORS/CLUC 2018 I decided to talk about device tree in Linux kernel, so here are blurb, presentation and video of that lecture.

You have one of those fruity *Pi arm boards and cheep sensor from China? Some buttons and LEDs? Do I really need to learn whole new scripting language and few web technologies to read my temperature, blink a led or toggle a relay?
No, because your Linux kernel already has drivers for them and all you need is device tree and cat.

Below is transcript of talk:

0:00:00.000,0:00:09.540
Hello. How are you?
This is the participation part, come on

0:00:09.540,0:00:19.619
you're not the first I time here!
OK, my name is Dobrica PavlinuÃÂ¡iÃÂ and I will try to

0:00:19.619,0:00:25.019
persuade you today that you can do with
your Linux something which you might not

0:00:25.019,0:00:32.969
have thought of by yourself I hope in a
sense in last year and a half I noticed

0:00:32.969,0:00:38.610
that I am using microcontrollers for
less and less and that I'm using my

0:00:38.610,0:00:44.760
Linux more and more for more or less the
same tasks and in that progress process

0:00:44.760,0:00:51.239
I actually learned something which I
want to share with you today in a sense

0:00:51.239,0:00:57.059
my idea is to tell you how to do
something with your arm single board

0:00:57.059,0:01:05.729
computer in this lecture we will talk
mostly about Allwinner boards but if

0:01:05.729,0:01:12.119
you want a hint if you want to buy
some arm computer please buy the board

0:01:12.119,0:01:17.070
which is supported by armbian. armbian  is the project which actually

0:01:17.070,0:01:21.360
maintains the distribution for our
boards and is currently the best

0:01:21.360,0:01:27.689
distribution for arms aside from me
raspbian for Raspberry Pi but raspbian

0:01:27.689,0:01:32.909
supports only Raspberry Pi but if you have
any other board please take a look if

0:01:32.909,0:01:37.860
there is an armbian port. if there isn't
try to contribute one and if you are

0:01:37.860,0:01:41.850
just deciding which board to buy my
suggestion is buy the one which is

0:01:41.850,0:01:47.070
already supported on the other hand if
you already did something similar you

0:01:47.070,0:01:52.079
might have found some references on the
internet about device three and it

0:01:52.079,0:01:55.619
looked like magic
so we'll try to dispel some of that

0:01:55.619,0:02:03.180
magic today unfortunately when you start
playing with it one of the first things

0:02:03.180,0:02:10.459
you will want to do is recompile the
kernel on your board so be prepared to

0:02:10.459,0:02:15.810
compile additional drivers if they are
not already included. armbian again

0:02:15.810,0:02:21.890
wins because it comes with a lot of
a lot of drivers already included and

0:02:21.890,0:02:28.950
this year I will not say anything which
requires soldering which might be good

0:02:28.950,0:02:35.190
for you if you are afraid of the heat
but it will be a little bit more than

0:02:35.190,0:02:41.519
just connecting few wires not much more
for a start let's start with the warning

0:02:41.519,0:02:46.980
for example you have a arms in arms
small arm board and you want to have a

0:02:46.980,0:02:52.019
real clock in it you know the one which
keeps the time when the board is powered

0:02:52.019,0:02:57.239
off has the battery and so on if you buy
the cheapest one from China which is

0:02:57.239,0:03:02.879
basically for Arduino you will buy the
device which is 5 volt device which your

0:03:02.879,0:03:10.170
arm single board computer isn't. you can
modify the board removing two resistors

0:03:10.170,0:03:16.409
if you want to but don't tell anyone I2C, and will mostly talk about i2c

0:03:16.409,0:03:23.819
sensors here, should be 5 volt tolerant
so if you by mistake just connected and

0:03:23.819,0:03:30.569
your data signals are really 5 volt you
won't burn your board but if you are

0:03:30.569,0:03:35.129
supplying your sensor with 5 volts
please double-check that your that is

0:03:35.129,0:03:39.060
sure to connect it to your board and
nothing bad will happen this is the only

0:03:39.060,0:03:44.970
warning I have for the whole lecture in
this example I showed you a really

0:03:44.970,0:03:49.829
simple way in which you can take the
sensors run i2cdetect, detect its

0:03:49.829,0:03:59.370
address in this case it's 68 - and then
- load 1 kernel module and all of the

0:03:59.370,0:04:03.889
sudden your Raspberry Pi will have battery backed clock, just like your laptop does

0:04:03.889,0:04:12.569
but how did this journey all started for
me? about two years ago I was very

0:04:12.569,0:04:18.060
unsatisfied with the choice of pinouts
which you can download from the internet

0:04:18.060,0:04:23.820
I was thinking something along the lines
wouldn't be it wouldn't it be nice if I

0:04:23.820,0:04:27.060
could
print the pinout for any board I have

0:04:27.060,0:04:36.360
with perfect 2.54 millimeters pin
spacing which I can put beside my pins

0:04:36.360,0:04:43.440
and never make a mistake of plugging the
wire in the wrong pin and we all know

0:04:43.440,0:04:48.000
that plugging the wire in the wrong pin
is always the first problem you have on

0:04:48.000,0:04:54.120
the other hand you say oh this is the
great idea and you are looking at your

0:04:54.120,0:04:58.320
pin out which is from the top of the
board and you are plugging the wires

0:04:58.320,0:05:02.160
from the bottom of the board and all of
the sudden your pin out has to be

0:05:02.160,0:05:06.660
flipped but once you write a script
which actually displays the pin out it's

0:05:06.660,0:05:12.950
trivially easy to get to add options to
flip it horizontally or vertically and

0:05:12.950,0:05:18.870
create either black and white pin out if
you are printing it a laser or color pin

0:05:18.870,0:05:26.639
out if you are printing it on some kind
of inkjet so once you have that SVG

0:05:26.639,0:05:31.680
which you can print and cut with the
scissors and so on it's just a script on

0:05:31.680,0:05:38.370
your machine so you could also have the
common line output and then it went all

0:05:38.370,0:05:44.070
south I started adding additional data
which you can see on this slide in

0:05:44.070,0:05:50.250
square brackets with the intention of
having additional data for each pin for

0:05:50.250,0:05:56.250
example if I started SPI I want to see
that this pin is already used so I will

0:05:56.250,0:06:01.830
not by mistake plug something into the
SPI pins if I already have the SPI pin

0:06:01.830,0:06:08.150
started if I have the serial port on
different boards your serial might be

0:06:08.150,0:06:14.880
UART4 on this particular CPU but it's
the only serial in your Linux system so

0:06:14.880,0:06:21.270
it will be /dev/ttyS0 for
example so I wanted to see all the data

0:06:21.270,0:06:27.180
and in the process I actually saw a lot
of things which kernel know and I didn't

0:06:27.180,0:06:33.690
so today talking to you about it of
course in comment line because you might

0:06:33.690,0:06:36.990
rotate your board well plug in the wires
you can also do

0:06:36.990,0:06:45.540
all the flips and things you you already
saw in the in the graphic part so let's

0:06:45.540,0:06:50.190
start with the sensor okay I said cheap
sensor for eBay we'll get the cheap

0:06:50.190,0:06:54.840
sensors from eBay but this sensor is from
some old PowerPC Macintosh. It was

0:06:54.840,0:06:59.580
attached to the disk drive and the
Macintosh used it to measure the

0:06:59.580,0:07:04.890
temperature of the disk drive. you know
that was in the times before the smart

0:07:04.890,0:07:12.060
had a temperature and I said hmm this
is the old sensor, kernel surely

0:07:12.060,0:07:16.800
doesn't have support for it, but, oh look,
just grep through the kernel

0:07:16.800,0:07:21.900
source and indeed there is a driver and
this was a start I said hmm

0:07:21.900,0:07:29.580
driver in kernel, I don't have to use
Arduino for it - now that I know that

0:07:29.580,0:07:35.070
driver is there and I have kernel module compiled, we said that prerequisite

0:07:35.070,0:07:40.950
is that we can compile the kernel, what
do we actually have to program or do to

0:07:40.950,0:07:46.260
make this sensor alive? not more than
this a echo in the middle of the slide

0:07:46.260,0:07:52.200
you just echo the name of the module and
the i2c address the i2c addresses we saw

0:07:52.200,0:07:56.070
it before we can get it with i2cdetect by just connecting the sensor and

0:07:56.070,0:08:02.280
the new device will magically appear it
will be shown in the sensors if you have

0:08:02.280,0:08:06.840
lm-sensors package installed but if
you don't you can always find the same

0:08:06.840,0:08:14.160
data in /sys/ file system which is full
of wonders and as we'll see in non

0:08:14.160,0:08:19.350
formatted way so the first two digits
actually the last three digits digits

0:08:19.350,0:08:26.640
are the decimal numbers and the all the
other are the the integer celsius in

0:08:26.640,0:08:34.470
this case. but you might say - I
don't want to put that echo in my startup

0:08:34.470,0:08:40.380
script! or I would like to have that as
soon as possible I don't want to depend

0:08:40.380,0:08:43.980
on the userland
to actually start my sensor and believe

0:08:43.980,0:08:48.710
it or not because your smart phones have
various sensors in

0:08:48.710,0:08:52.790
them, there is a solution in the Linux kernel
for that and it's called the device tree

0:08:52.790,0:08:58.700
so this is probably the simplest form of
the device tree which still doesn't look

0:08:58.700,0:09:06.020
scary, but it i will, stay with me, and it
again defines our module the address

0:09:06.020,0:09:12.020
which is 49 in this case and i2c 1
interface just like we did in that echo

0:09:12.020,0:09:16.700
but in this case this module will be
activated as soon as kernel starts up

0:09:16.700,0:09:24.590
as opposed to the end of your boot up
process. one additional thing that kernel

0:09:24.590,0:09:31.000
has and many people do not use is
ability to load those device trees

0:09:31.000,0:09:35.750
dynamically the reason why those most
people don't use it is because they are

0:09:35.750,0:09:41.270
on to old kernels I think you have to
have something along the lines of 4.8

0:09:41.270,0:09:47.510
4.8 or newer to actually have the
ability to load the device trees live

0:09:47.510,0:09:54.470
basically you are you are using /sys/kernel/config directory and this script

0:09:54.470,0:10:00.620
just finds where you have it mounted and
loads your device tree live word of

0:10:00.620,0:10:09.200
warning currently although it seems like
you can do that on Raspberry Pi the API

0:10:09.200,0:10:14.270
is there the model is compiled,
everything is nice and Diddley, kernel

0:10:14.270,0:10:18.350
even says that device tree overlay is
applied, it doesn't work on the raspberry

0:10:18.350,0:10:24.950
pi because raspberry pi is different but
if you gave a any other platform live

0:10:24.950,0:10:31.670
loading is actually quite nice and
diddley. so now we have some sensor and it

0:10:31.670,0:10:37.310
works or it doesn't
and we somewhat suspect that kernel

0:10:37.310,0:10:43.040
developers didn't write a good driver
which is never ever the case if you

0:10:43.040,0:10:48.260
really want to implement some driver
please look first at the kernel source

0:10:48.260,0:10:52.220
tree there probably is the
implementation better than the one you

0:10:52.220,0:10:56.930
will write and you can use because the
kernel is GPL you can use that

0:10:56.930,0:11:00.970
implementation as a reference because in
my

0:11:00.970,0:11:05.890
small experience with those drivers in
kernel they are really really nice but

0:11:05.890,0:11:10.960
what do you do well to debug it?
I said no soldering and I didn't say but

0:11:10.960,0:11:15.010
it would be nice if I could do that
without additional hardware. Oh, look

0:11:15.010,0:11:20.890
kernel has ability to debug my i2c
devices and it's actually using tracing.

0:11:20.890,0:11:26.140
the same thing I'm using on my servers
to get performance counters. isn't that

0:11:26.140,0:11:30.340
nice. I don't have to have a logic
analyzer. I can just start tracing and

0:11:30.340,0:11:40.120
have do all the dumps in kernel. nice,
unexpected, but nice! so let's get to the

0:11:40.120,0:11:45.790
first cheap board from China so you
bought your arm single base computer

0:11:45.790,0:11:52.030
single board computer and you want to
add few analog digital converters to it

0:11:52.030,0:11:57.070
because you are used to Arduino and you
have some analog sensor or something and

0:11:57.070,0:12:05.620
you found the cheapest one on eBay and
bought few four five six because they

0:12:05.620,0:12:10.990
are just the $ each, so what do you do
the same thing we saw earlier you just

0:12:10.990,0:12:17.800
compile the models say the address of
the interface and it will appear and

0:12:17.800,0:12:23.770
just like it did in the last example so
everything is nice but but but you read

0:12:23.770,0:12:30.790
the datasheet of that sensor and the
sensor other than 4 analog inputs also

0:12:30.790,0:12:38.860
has 1 analog output which is the top
pin on the left denoted by AOUT, so you

0:12:38.860,0:12:42.850
want to use it
oh this kernel model doesn't is not very

0:12:42.850,0:12:46.900
good it doesn't have ability to control
that of course it does but how do you

0:12:46.900,0:12:50.680
find it?
my suggestion is actually to search

0:12:50.680,0:12:57.190
through the /sys/ for either address of
your i2c sensor, which is this

0:12:57.190,0:13:05.790
case is 48, or for the word output or
input and you will actually get all

0:13:05.790,0:13:12.400
files, because in linux everything is a
file, which are defined in driver of this

0:13:12.400,0:13:16.620
module, and if you
look at it, there is actually out0_output

0:13:16.620,0:13:24.630
out0_output file in which you can
turn output on or off so we are all

0:13:24.630,0:13:29.910
golden. kernel developers didn't forget
to implement part of the driver for this

0:13:29.910,0:13:38.910
sensor all golden my original idea was
to measure current consumptions of arm

0:13:38.910,0:13:44.490
boards because I'm annoyed by the random
problems you can have just because your

0:13:44.490,0:13:49.470
power supply is not powerful enough so
it's nice actually to monitor your power

0:13:49.470,0:13:55.740
usage so you will see what changes, for
example, you surely... we'll get that, remind

0:13:55.740,0:14:01.250
me to tell you how much power does the
 the additional button take

0:14:01.250,0:14:05.130
that's actually interesting thing which
you wouldn't know if you don't measure

0:14:05.130,0:14:10.950
current so you buy the cheapest possible
eBay sensor for current, the right one

0:14:10.950,0:14:16.560
is the ina219 which is
bi-directional current sensing so you

0:14:16.560,0:14:21.690
can put it between your battery and solar panel and you will see

0:14:21.690,0:14:26.160
whether the battery is charging or
discharging, or if you need more channels

0:14:26.160,0:14:34.020
I like an ina3221 which has 3 channels,
the same voltage

0:14:34.020,0:14:40.500
but 3 different channels, so you can
power 3 arm single computers from one

0:14:40.500,0:14:47.670
sensor if you want to. and of course once
you have that again the current is in

0:14:47.670,0:14:53.260
some file and it will be someting...

0:14:53.260,0:14:56.490
But, I promised you IOT? right? nothing

0:14:56.490,0:14:59.070
I said so far is IOT! where is the
Internet?

0:14:59.070,0:15:05.100
where are the things? buzzwords are missing! OK, challenge

0:15:05.100,0:15:08.550
accepted! Let's make a button! you know it's like a

0:15:08.550,0:15:15.029
blink LED. So buttons, because I was
not allowed to use soldering iron in

0:15:15.029,0:15:21.990
this talk, I'm using old buttons from old
scanner. nothing special 3 buttons, in

0:15:21.990,0:15:25.350
this case with hardware debounce, but we
don't care.

0:15:25.350,0:15:29.270
4 wires 3 buttons.
how hard can it be?

0:15:29.270,0:15:36.240
well basically it can be really really
simple  this is the smallest

0:15:36.240,0:15:42.638
font so if you see how to read this I
congratulate you! in this case I am

0:15:44.580,0:15:51.360
specifying that I want software pull up, in this first fragment on the top.

0:15:51.360,0:15:57.840
I could have put some
resistors, but you said no soldering

0:15:57.840,0:16:03.540
so here I am telling to my
processor. please do pull up on

0:16:03.540,0:16:09.540
those pins. and then I'm defining three
keys. as you can see email. connect and

0:16:09.540,0:16:15.660
print. which generate real Linux
keyboard events. so if you are in X and

0:16:15.660,0:16:20.730
press that key, it will generate that key,
I thought it would it would be better to

0:16:20.730,0:16:26.400
generate you know the magic multimedia
key bindings as opposed to A, B and C

0:16:26.400,0:16:31.940
because if I generated a ABC and was its
console I would actually generate

0:16:31.940,0:16:39.140
letters on a login prompt which I didn't
want so actually did it and it's quite

0:16:39.140,0:16:47.310
quite easy. In this case I'm using gpio-keys-polled which means that my CPU

0:16:47.310,0:16:52.500
is actually pulling every 100
milliseconds those keys to see whether

0:16:52.500,0:16:57.150
they their status changed and since the
board is actually connected through the

0:16:57.150,0:17:01.890
current sensing - sensor I mentioned earlier I'm

0:17:01.890,0:17:11.209
getting additional: how many mA?
every 100 milliseconds, pulling 3 keys?

0:17:13.010,0:17:19.350
60! I wouldn't expect my power
consumption to rise by 60 milliamps

0:17:19.350,0:17:24.480
because I am pulling 3 keys every 100
milliseconds! but it did and because I

0:17:24.480,0:17:32.430
could add sensors to the linux without
programming drivers, I know that! why am I

0:17:32.430,0:17:40.450
using polling because on allwinner
all pins are not interrupt capable so in

0:17:40.450,0:17:44.520
the sense all pins cannot generate
interrupts on allwinner

0:17:44.520,0:17:48.940
raspberry pi in this case is different
on raspberry pi every pin can be

0:17:48.940,0:17:54.279
interrupt pin on Allwinner that is not
the case. so the next logical question is

0:17:54.279,0:17:58.510
how do I know when I'm sitting in front
of my board whether the pin can get the

0:17:58.510,0:18:05.440
interrupt or not? my suggestion is ask
the kernel. just grap through the debug

0:18:05.440,0:18:10.720
interface of the kernel through pinctrl
which is basically the the thing

0:18:10.720,0:18:19.570
which configures the pins on your arm
CPU and try to find the irq

0:18:19.570,0:18:25.840
will surely get the list of the pins
which are which are irq capable take in

0:18:25.840,0:18:30.370
mind that this will be different on
different arm architectures so

0:18:30.370,0:18:34.419
unfortunately on allwinner
it will always look the same because it

0:18:34.419,0:18:41.020
is allwinner architecture. actually
sunxi ! but on the Raspberry Pi for

0:18:41.020,0:18:45.880
example this will be somewhat different
but the kernel knows, and the grep is

0:18:45.880,0:18:51.490
your friend. so you wrote the device
three you load it either live or some

0:18:51.490,0:18:57.279
something on some other way you connect
your buttons and now let's try does it really

0:18:57.279,0:19:03.580
work? of course it does! we will start
evtest which will show us all the

0:19:03.580,0:19:08.350
input devices we have the new one is the
gpio-3-buttons, which is the same name

0:19:08.350,0:19:14.559
as our device three overlay and we can see
 the same things we saw in

0:19:14.559,0:19:20.230
device three we defined three keys with
this event but we free-of-charge

0:19:20.230,0:19:25.990
got for example keyboard repeat because
this is actually meant to be used for

0:19:25.990,0:19:31.630
keyboards our kernel is automatically
implementing repeat key we can turn it

0:19:31.630,0:19:35.919
off but this is example of one of the
features which you probably wouldn't

0:19:35.919,0:19:43.330
implement yourself if you are connecting
those three keys to your Arduino but but

0:19:43.330,0:19:46.120
but this is still not the
internet-of-things

0:19:46.120,0:19:49.789
if this is the Internet it should have
some kind of

0:19:49.789,0:19:56.299
Internet in it some buzzwords for
example mqtt and it really can just

0:19:56.299,0:20:01.759
install trigger-happy demon which is
nice deamon which listens to

0:20:01.759,0:20:06.830
input events and write a free file
configuration which will send the each

0:20:06.830,0:20:15.139
key pressed order MQTT and job done i
did the internet button without a line

0:20:15.139,0:20:23.419
of code the configuration one side note
here if you are designing some some

0:20:23.419,0:20:30.919
Internet of Things thingy which even if
you are only one who will use it it's a

0:20:30.919,0:20:33.889
good idea but if you are doing it for
somebody else

0:20:33.889,0:20:41.119
please don't depend on the cloud because
I wouldn't like for my door to be locked

0:20:41.119,0:20:45.590
permanently with me outside just because
my internet connection isn't working

0:20:45.590,0:20:52.999
think about it of course you can use any
buttons in this case this was the first

0:20:52.999,0:20:57.070
try actually like the three buttons
better than this one that's why this

0:20:57.070,0:21:02.419
these buttons are coming second and this
board with the buttons has one

0:21:02.419,0:21:08.840
additional nice thing and that is the
LED we said that will cover buttons and

0:21:08.840,0:21:15.379
LEDs right unfortunately this LED is 5
volts so it won't light up on 3.3 volts

0:21:15.379,0:21:24.139
but when you mentioned LEDs something
came to mind how can I use those LEDs

0:21:24.139,0:21:29.029
for something more useful than just
blinking them on or off we'll see you

0:21:29.029,0:21:34.039
later then turning them on or off it is
also useful you probably didn't know

0:21:34.039,0:21:40.580
that you can use Linux triggers to
actually display status of your MMC card

0:21:40.580,0:21:47.720
CPU load network traffic or something
else on the LEDs itself either the LEDs

0:21:47.720,0:21:51.859
which you already have on the board but
if your manufacturer didn't provide

0:21:51.859,0:21:56.679
enough of them you can always just add
random LEDs, write device tree and

0:21:56.679,0:22:01.559
define the trigger for them, and this is
an example of that

0:22:01.559,0:22:08.219
and this example is actually for this
board which is from the ThinkPad

0:22:08.219,0:22:14.019
ThinkPad dock to be exact, which
unfortunately isn't at all visible

0:22:14.019,0:22:20.169
on this picture, but you will believe me,
has actually 2 LEDs and these 3

0:22:20.169,0:22:27.489
keys and 2 LEDs actually made the arm
board which doesn't have any buttons on

0:22:27.489,0:22:36.609
it or status LEDs somewhat flashy with
buttons. that's always useful on the

0:22:36.609,0:22:41.889
other hand here I would just want to
share a few hints with you for a start

0:22:41.889,0:22:47.320
first numerate your pins because if you
compared the the picture down there

0:22:47.320,0:22:54.579
which has seven wires and are numerated
which is the second try be the first

0:22:54.579,0:23:00.579
nodes on the up you will see that in
this case i thought that there was eight

0:23:00.579,0:23:06.999
wires so the deducing what is connected
where when you have the wrong number of

0:23:06.999,0:23:14.229
wires is maybe not the good first step
on the other hand we saw the keys what

0:23:14.229,0:23:21.639
about rotary encoders? for years I was
trying to somehow persuade Raspberry Pi

0:23:21.639,0:23:28.149
1 as the lowest common denominator of
all arm boards you know cheap slow and

0:23:28.149,0:23:33.789
so on to actually work with this exact
rotary encoder the cheapest one from the

0:23:33.789,0:23:41.589
Aliexpress of course see the pattern
I tried Python I try the attaching

0:23:41.589,0:23:48.579
interrupt into Python I tried C code and
nothing worked at least didn't work

0:23:48.579,0:23:56.859
worked reliably and if you just write
the small device tree say the correct

0:23:56.859,0:24:01.809
number of steps you have at your rotary
encoder because by default it's 24 but

0:24:01.809,0:24:08.589
this particular one is 20 you will get
perfect input device for your Linux with

0:24:08.589,0:24:11.579
just a few wires

0:24:12.390,0:24:20.260
amazing so we saw the buttons we saw the
LEDs we have everything for IOT except

0:24:20.260,0:24:25.840
the relay so you saw on one of the
previous pictures this relay box it's

0:24:25.840,0:24:31.980
basically four relays separated by
optocouplers which is nice and my

0:24:31.980,0:24:39.670
suggestion since you can't in device
three you can't say this pin will be

0:24:39.670,0:24:45.790
output but I want to initially drive it
high or I want to initially drive it low

0:24:45.790,0:24:52.330
it seems like you can say that it's
documented in documentation it's just

0:24:52.330,0:24:56.440
not implemented there on every arm
architecture so you can write it in

0:24:56.440,0:25:02.440
device three but your device tree will
ignore it. you but you can and this might be

0:25:02.440,0:25:07.240
somewhat important because for example
this relay is actually powering all your

0:25:07.240,0:25:11.220
other boards and you don't want to
reboot them just because you reboot the

0:25:11.220,0:25:16.270
machine which is actually driving the
relay so you want to control that pin as

0:25:16.270,0:25:24.310
soon as possible so my suggestion is
actually to explain to Linux kernel that

0:25:24.310,0:25:30.310
this relays is actually 4 LEDs which is
somewhat true because the relay has the

0:25:30.310,0:25:36.460
LEDs on it and then use LEDs which do
have the default state which works to

0:25:36.460,0:25:41.410
actually drive it as soon as possible as
the kernel boot because kernel will boot

0:25:41.410,0:25:45.640
it will change the state of those pins
from input to output and set them

0:25:45.640,0:25:52.000
immediately to correct value so you
hopefully want want power cycle your

0:25:52.000,0:25:56.410
other boards, and then you can use the
LEDs as you would normally use them in

0:25:56.410,0:26:01.420
any other way
if LEDs are interesting to you have in

0:26:01.420,0:26:06.340
mind that on your on each of your
computers you have at least two LEDs but

0:26:06.340,0:26:10.810
this caps lock and one is non lock on
your keyboard and you can use those same

0:26:10.810,0:26:15.190
triggers I mentioned earlier on your
existing Linux machine using those

0:26:15.190,0:26:20.980
triggers so for example your caps lock
LED can blink as your network traffic

0:26:20.980,0:26:29.380
does something on your network really
it's fun on the other hand if you have a

0:26:29.380,0:26:36.370
Raspberry Pi and you defined everything
correctly you might hit into some kind

0:26:36.370,0:26:42.370
of problems that particular chip has
default pull ups which you can't turn on

0:26:42.370,0:26:49.059
for some pins which are actually
designed to be clocks of this kind or

0:26:49.059,0:26:54.220
another so even if you are not using
that pin as the SPI clock whatever you

0:26:54.220,0:26:59.110
do in your device tree you won't be able to
turn off, actually you will turn

0:26:59.110,0:27:04.240
off the the setting in the chip to for
the pull up but the pull up will be

0:27:04.240,0:27:08.140
still there actually thought that there
is a hardware resistor on board but

0:27:08.140,0:27:14.980
there isn't it's inside the chip just
word of warning so if anything I would

0:27:14.980,0:27:21.309
like to push you towards using Linux
kernel for the sensors which you might

0:27:21.309,0:27:25.570
not think of as the first choice if you
just want to add some kind of simple

0:27:25.570,0:27:32.080
sensor to to your Linux instead of
Arduino over serial port which I did and

0:27:32.080,0:27:38.049
this is the solution which might with
much less moving parts or wiring pie and

0:27:38.049,0:27:43.090
in the end once you make your own shield
for us but if I you will have to write

0:27:43.090,0:27:49.480
that device tree into the EEPROM anyway so
it's good to start learning now so I

0:27:49.480,0:27:53.890
hope that this was at least useful very
interesting and if you have any

0:27:53.890,0:28:02.140
additional questions I will be glad to
to answer them and if you want to see

0:28:02.140,0:28:07.780
one of those all winner board which can
be used with my software to show the pin

0:28:07.780,0:28:12.460
out here is one board which kost
actually borrowed me yesterday and in which

0:28:12.460,0:28:18.549
yesterday evening I actually ported the
my software which is basically just

0:28:18.549,0:28:22.330
writing the definition of those pins
over here and the pins on the header

0:28:22.330,0:28:26.950
which I basically copy pasted from the
excel sheet just to show that you can

0:28:26.950,0:28:32.830
actually do that for any board you have
with just really pin out in textual file

0:28:32.830,0:28:36.370
it's really that simple
here are some additional

0:28:36.370,0:28:38.300
links and do you have any questions?

0:28:39.420,0:28:41.420
[one?]

How to have a higher chance of success when restoring a big MySQL database

August 13, 2017 07:20

Restoring a MySQL database is fast and easy when you just copy files in datadir when the server is shutdown, or if you use Percona xtrabackup.

But if you for some reason (AWS RDS) only have MySQL protocol available for backup, you usually can have a compressed mysqldump, that is quite slow to restore, not because of the compression or because the decompressed version is a text file that needs to be parsed, but because MySQL is slow to push it through it's disk pipeline, and because it needs to build data indexes while doing a restore.

I've spent multiple days babysitting the process of restoring a 7GB gzip compressed MySQL dump file, and these are results and tips that could help you save some time.

So, make sure that:
- you have enough IO available: For restoring a 66 GB datadir 315.6 GB was written to the drive (as measured with iostat), with a tuned MySQL configuration. For a DB of this size a mechanical drive doesn't cut it, and restore will take multiple days. Use a good SDD.

- your database TRIGGERS all have BEGIN/END statements (even though you can create them without and even thought the bug was supposed to be fixed https://bugs.mysql.com/bug.php?id=16878), it fails on restore, with all versions of MySQL 5.7/5.6 i tried

- you start with a really empty database in your datadir - DB I worked with had inconsistent data types on a foreign key, when the dependent table with an inconsistent key already exists MySQL will report a foreign key error (MariaDB will be more informative), but if it doesn't it will happily restore the database

- your max_allowed_packet conf value is big enough, or you'll get a MySQL server has gone away message from your client while restoring.

- your innodb_log_file_size is big enough (https://dev.mysql.com/doc/refman/5.6/en/innodb-parameters.html#sysvar_innodb_log_file_size) - if you have large BLOB values in your DB, restore will fail if the value is lower than 10% of your blob field. This setting is important for quick restore too

- you have log-bin turned off in order to minimize your chance to run out of drive space and save IO (log-bin=Off doesn't mean that it's disabled just that the log bin files start with Off, the documentation can be confusing here :) What worked for me is having all log-bin lines in the mysqld config section commented out

Finally, if you want it to finish quickly, use the fastest SSD you have available, and consider tuning MySQL configuration a bit. I'm also considering using a ramdisk, because it would help both with restore speed and when you need to do some DB transformations. MySQL defaults are not reasonable, especially for innodb_log_file_size, max_allowed_packet.

I used excellent pv for figuring out if the restore process will finish in reasonable time

pv db_dump.gz |gunzip -c |mysql -uroot database_name

Here's a full list of my MySQLd configuration variables that worked for me on my dev laptop

#my dev laptop is low on memory, for prod server you would use a lot more
innodb_buffer_pool_size=512M
innodb_additional_mem_pool_size=256M
innodb_log_buffer_size=256M
innodb_log_file_size=512M
max_allowed_packet=64M

#for saving disk IO, dont use on prod
innodb_flush_log_at_trx_commit = 2
innodb_flush_method=O_DIRECT_NO_FSYNC
skip-innodb_doublewrite

Some thoughts on (Modern) PHP

July 30, 2017 13:45

I have experience in both Java and PHP. Java mostly for traditional desktop apps, embedded UIs and PHP for websites.

Custom PHP frameworks I've built or helped build took into account the way PHP is executed: you are stateless and need to setup everything on every request (runtime is fast to start with FPM and opcode cache). Namespaces based cheap autoloader worked great. We used singletons for getting the configuration and connections to DBs. There was almost no setup code that needed to be run every time other than loading .ini based configuration and connecting to the DB. My webapps responded under 20ms (DB and other services like sphinx included), and I could get it to respond in 1 ms for things where we needed to be quick and didn't have to output HTML with Forms. It was really small and you could read the whole framework code in 1-2 hours. It worked with SQL in a reasonable way. You didn't have to write your SQL for simple CRUD, but for larger things involving joining multiple tables and more complex expressions we wrote native SQL. Caching was done thoughtfully, using APC user cache (SHM with zero copy). It just felt nice.

I switched jobs recently, and started with Symfony 3. The thing felt like some Java framework, but poorly documented and harder to use than it should be. It had lots and lots of setup code done before handling every request. There's a whole DI framework with it's load of setup code for every component. And you have to do setup even though you don't use the component in that particular request. There are ways of doing setup lazily, but you still waste time to wire that up. Framework overhead can be 30-100ms. Other modern PHP frameworks often have similar overhead. I know that there's PHP-PM, to save some of that work that isn't really $_REQUEST specific, but it doesn't seem to be used much for production. And using Silex (deprecated by Symfony 4?) is really not that different, you still either reuse Symfony components, or rewrite them, but with similar "best practices" that are inspired by Java.

Regarding persistence, Doctrine and it's verbosity feels very ugly to me. I'd much rather use SQL syntax for defining relationships than bunch of PHP with special syntax comments or xml or yaml. And also use real SQL for complex queries.

Everybody is using type hints wherever they can, and it feels as verbose as Java, but without compile time type safety, and you can't really put type information everywhere (class members for example).

So you are almost using a type safe language, but can't get performance or compile time benefits, because inevitably, you'll have to use some dynamic typing or other dynamic language features.

Even though PHP runtime has made great progress with 7.x (it's probably the fastest interpreted language, and it's great it has reference counted garbage collection), it feels like language is struggling to find it's identity, with it taking a lot from Java and still coping with ugly legacy ($, having to use $this inside a class function, php.ini, features for supporting templating even though it's rarely used as a templating language in modern frameworks, https://3v4l.org/mcpi7).

Learning Python and Flask (as an example) was much more enjoyable than switching from a nimble custom PHP framework to Symfony. Using NodeJS and minimalistic components to build my own framework was also nice. I'd love to try GoLang, Swift or Rust in the backend too.

And there's the thing that most of the PHP frameworks try too hard to be full stack, when nowdays it's not rare you only do REST APIs on the backend. So there's a lot of infrastructure and assumptions in place for rendering HTML that you really don't need to use and that gets in the way when learning the framework and is wasteful when the code is executing.

I'd argue you can write fast, simple and maintainable PHP, by using state PHP runtime has setup for you ($_POST, $_GET, $_SERVER etc), namespaces and a namespace based autoloader, trying to use pure functions when you can (using static classes shouldn't be a sin - use it to split your code in sensible parts), and using general good practices for writing readable and maintainable code (avoid long functions, huge classes, too much block nesting, decoupling, naming things in a good way). With some coding conventions you can write a decent and productive framework quickly, but you could do that with a nicer language too, so what's the point?

(Thankfully, I'm not using Symfony on my new job, and Yii2 does suffer from some issues too, it at least feels better for now)

etckeeper, bind jnl files and git-pack memory problems

July 22, 2016 13:57

For last few years, one of first tools which we install on each new server is etckeeper. It saved us couple of times, and provides nice documentation about changes on the system.

However, git can take a lot of space if you have huge files which change frequently (at least daily since etckeeper has daily cron job to commit changes done that day). In our case, we have bind which stores jnl files in /etc/bind which results in about 500 Kb change each day for 11 zones we have defined.

You might say that it doesn't seem to be so bad, but in four months, we managed to increase size of repository from 300 Mb to 11 Gb. Yes, this is not mistake, it's 11000 Mb which is increase of 36 times! Solution for this is to use git gc which will in turn call git-pack to compress files. But this is where problems start -- git needs a lot of RAM to do gc. Since this machine has only 1 Gb of RAM, this is not enough to run git gc without running out of memory.

Last few times, I transferred git repository to other machine, run git gc there and than transfered it back (resulting in nice decrease from 11 Gb back to 300 Mb), however, this is not ideal solution. So, let's remove bind jnl files from etckeeper...

Let's start with our 11 Gb git repo, copy it to another machine which has 64 Gb or RAM needed for this operation.

root@dns01:/etc# du -ks .git
11304708        .git

root@dns01:~# rsync -ravP /etc/.git build.ffzg.hr:/srv/dns01/etc/

Now, we will re-create local files because we need to find out which jnl files are used so we can remove them from repo.

root@build:/srv/dns01/etc# git reset --hard

ls bind/*.jnl | xargs -i git filter-branch -f --index-filter 'git rm --cache --ignore-unmatch {}'

echo 'bind/*.jnl' >> .gitignore
git commit -m 'ignore ind jnl files' .gitignore

Now, finally we can shrink our 11 Gb repo!

root@build:/srv/dns01/etc# du -kcs .git
11427196        .git

root@build:/srv/dns01/etc# git gc
Counting objects: 38117, done.
Delta compression using up to 18 threads.
Compressing objects: 100% (27385/27385), done.
Writing objects: 100% (38117/38117), done.
Total 38117 (delta 27643), reused 12846 (delta 10285)
Removing duplicate objects: 100% (256/256), done.

root@build:/srv/dns01/etc# du -ks .git
414224  .git

# and now we can copy it back...

root@dns01:/etc# rsync -ravP build.ffzg.hr:/srv/dns01/etc/.git .

Just as side note, if you want to run git gc --aggressive on same repo, it won't finish with 60 Gb or RAM and 100 Gb of swap, which means that it needs more than 150 Gb of RAM.

So, if you are storing modestly sized files which change a lot, have in mind that you might need more RAM to run git gc (and get disk usage under control) than you might have.

Let's hack cheap hardware - 2016 edition

May 17, 2016 10:11

Last week I head pleasure to present at two conferences in two different cities: DORS/CLUC 2016 and Osijek Mini Maker Faire on topic of cheap hardware from China which can be improved with little bit of software or hardware hacking. It was well received, and I hope thet you will find tool or two in it which will fill your need.

I hope to see more hacks of STM8 based devices since we have sdcc compiler with support for stm8, cheap SWIM programmer in form of ST-Link v2 (Chinese clones, which are also useful as ARM SWD programmers) and STM8 has comparable features to 8-bit AVR micro-controllers but cheaper.

Kabelski internet i oversubscription

May 14, 2016 16:50

Ovo je post iz 5.11.2014. U međuvremenu sam promjenio kabelskog operatera

Ako imate kabelski internet to znači da najvjerovatnije koristite jednu od sljedećih kabelskih tehnologija za prijenos digitalnih podataka:

- DOCSIS 1.0, 1.1, 2.0, 3.0 ili EuroDOCSIS standardi

- PacketCable 1.0, 1.5, 2.0 standardi koji na DOCSIS bazi grade razne usluge poput telefonije i digitalne televizije

Frekvencijski pojas svakog kabela podjeljen je na kanale. Širina kanala ovisi o standardu pa tako EuroDOCSIS koristi europsku širinu kanala od 8 MHz , a DOCSIS koristi američku od 6 MHz.

Podjela bandwidtha koaksijalnog kabela (Maksimalni downstream bandwidth koaksijalnog kabela je 4864 megabita prema primjeru niže)

Svi spomenuti DOCSIS transportni standardi imaju slične karakteristike oko toga koliku downstream propusnost podržavaju po jednom megahertzu, pa tako DOCSIS podržava 38 megabita po kanalu downloada, a EuroDOCSIS 50 megabita po kanalu downloada.

DOCSIS 1.1 je donio bolju standardizaciju i mogućnosti kontroliranja kvalitete usluge (QoS)

DOCSIS 2.0 je donio bolje upload brzine (27 megabita po kanalu u odnosu na DOCSIS 1.0 9 megabita po kanalu)

DOCSIS 3.0 je donio mogućnost da jedan korisnik istovremeno koristi više kanala tako povećavajući bandwidth.

DOCSIS 3.1 izdan u Listopadu 2013. je prva veća promjena u standardu jer donosi novu modulaciju 4096 QAM i odustaje od podjele kanala na 6 ili 8 MHz i umjesto toga koristi manje OFDM podkanale i u idealnim uvjetima podržava brzine do 10 gigabita downstream i 1 gigabit upstream. Još nije u primjeni.

E sad, sve je to divno i krano, ali zašto je uz takve ogromne brojke moj internet spor?

Koaksijalni kabel je medij koji dijelimo sa drugim korisnicima, za razliku od DSL-a gdje svaki modem ima vlastitu bakrenu paricu do centrale, kod kabelskih mreža dijelimo medij sa neodređenim i samo vašem ISP-u poznatim brojem korisnika. Obično operater nudi i uslugu kabelske televizije te je prostor za vaš internet sužen sa brojem kanala koji se koriste za TV uslugu.

Ajmo vidjeti jedan primjer u praksi na zagrebačkom području, za downstream:

Motorola SBV5121E

Koristi se modem Motorola SBV5121E (DOCSIS 2.0 i niže), što prema specifikaciji [2] znači da ima bandwidth za downstream od 88 do 860 MHz sa američkom širinom kanala od 6 MHz. Znači 772/6 = 128 kanala. Operater koji sam analizirao po mom saznanju 40 analognih TV kanala i 113 digitalnih. Recimo da se za ovih 113 digitalnih troši 30 6 MHz kanala u kabelu. Što znači da recimo srijedu uvečer, kad se ljudi vrate sa posla i škole, samo 58 različitih kućanstava (kanala) može istovremeno surfati punom brzinom od 38 megabita, svaki sljedeći korisnik koji krene surfati smanjuje brzinu ovim ostalima.

Graf latencije (do prvog hop-a) na primjeru Zagrebačkog ISP-a dok korisnik osim za mjerenje ne koristi uslugu.

Operater kojeg sam analizirao nudi brzine od 8 megabita, što znači da bi teoretski trebao moći dati traženi bandwidth za (38/8) *58 = 275 korisnika, no pošto se tu vrijeme provedeno na kanalu po korisniku mora smanjiti kako bi se jedan kanal podjelio na više kućanstava, u tim slučajevima, čak i da surfa samo 275 kućanstava, njihova latencija (ICMP ping) sa odličnih 6-7 ms počinje rasti na (worst case, puna utilizacija na 418 korisnika) 4.75*7= 33 ms (molim ispravak ako je računica netočna, uzimam u obzir najmanju veličinu ICMP paketa tj. najmanju moguću diskretnu jedinicu u kojoj je moguće ostvariti komunikaciju).

Dodatni problem je što DOCSIS 2.0 i niži ne omogućavaju brzo prebacivanje među kanalima, što znatno otežava dobru iskoristivost frekventnog spektra kabela (možda na drugim kanalima ima značajno više prostora za prijenos podataka).

U svakom slučaju, ako je previše korisnika koji dijele isti resurs (isti 6 MHz kanal, isti kabel) dolazi do drastičnog brzine pristupa pa tako kod ISP-a koji sam analizrao bandwidth pada na ispod 1 megabita, a ping ide i iznad 140 ms, uz česti packet loss.

Rijetko kada svih korisnici žele istovremeno i na period dulji od nekoliko minuta maksimalni bandwidth, pa je moguće (prema brojkama u primjeru) imati 10x više korisnika nego što je ukupnog kapaciteta (npr. 418 korisnika na 8 megabita na 88 kanala nego 4180 korisnika) a da sami korisnici ne primjete probleme u brzini pristupa, ali to uvelike ovisi o načinu korištenja Interneta. Moguće je da će više učenja na daljinu, skidanja igara preko Steam-a i sličnih servisa itd. značajno promjeniti navike korisnika u budućnosti.

Posao dijeljenja bandwidtha kada je više korisnika od broja slobodnih kanala rade zajedno modem kod korisnika i CMTS uređaj kod operatera. CMTS radi mnoge slične funkcije koje u DSL sustavima radi DSLAM, ali uzevši u obzir karakteristike dijeljenog koaksijalnog medija. CMTS omogućava da i do 1000 korisnika dijeli isti 6 MHz kanal. Koristi tehniku zvanu Statistical time division multiplexing. Nisam našao na podatak može li jedan CMTS uređaj stvarno i napuniti svih 128 downstream kanala i još 60 Mhz upstream bandwidtha. Svakako mu za to treba barem 10 gbit ethernet sučelje.

ISP može poboljšati infrastrukturu tako da smanji broj korisnika koji dijele jedan jedini kabel, ili poveća broj kanala koji se koriste za DOCSIS ukoliko medij ima slobodne kanale.

Također, ISP može početi koristiti digitalnu TV kako bi iskoristio mogućnost digitalne kompresije video i audio zapisa i time smanjio potreban bandwidth po TV kanalu za bar 4 puta (moguće i više sa kompresijom naprednijom od MPEG2), no ovo znači da operater mora svim korisnicima zamjeniti receivere za TV, što može biti značajna investicija.

Osnovana je i Facebook grupa gdje se korisnici mogu požaliti na svog operatera ili raspravljati o boljim operaterima i tehnlogijama poput recimo FTTH ili VDSL-a.

Pridružite nam se na:

https://www.facebook.com/groups/hocuboljiinternet/

Linkovi:

[1] http://en.wikipedia.org/wiki/DOCSIS

[2] http://www.wiretechsa.com.ar/PDF/equipamientointernet/SBV5121.pdf

[3] http://computer.howstuffworks.com/cable-modem.htm

[4] http://www.lightreading.com/cable-video/docsis/docsis-31-whats-next/d/d-id/708425
[5] http://www.cisco.com/c/dam/en/us/solutions/collateral/service-provider/cable-high-speed-data-hsd-solutions/gateway_to_connected_life_white_paper.pdf

Smart public transport with small automated, semi-automated or manually driven vehicles

May 01, 2016 14:08

Here's just and idea (feel free to use it in any way):

Imagine having a network of small (4-6 passengers) vehicles servicing a city for daily transportation needs. Users would enter a desired location and arrival time. The arrival time could be flexible (within an hour, if not then the price could be appropriately higher) and the user would announce any regularity (for example detailing a weekly commute) that could be used for future planning.

The centralized system would optimize the problem of getting all passengers to their respective locations and suggest departure time and location (preferably within a few minutes of walking distance).

An interesting open source implementation would use OpenStreetMap data and have simulations and visualizations. A commercial entity could deal with deployments on various locations and provide a stable software as a service around the core open implementation. Autonomous vehicles would provide much more efficient operation of such a network and lower the costs significantly.

Debian OpenLDAP with GnuTLS and OpenSSL certificates

January 19, 2016 10:00

Every few years we have to renew SSL certificates. And there is always something which can go wrong. So I decided to reproduce exact steps here so that Google can find it for next unfortunate soul who has same problem.

Let's examine old LDAP configuration:

deenes:/etc/ldap/slapd.d# grep ssl cn\=config.ldif 
olcTLSCACertificateFile: /etc/ssl/certs/chain-101-mudrac.ffzg.hr.pem
olcTLSCertificateFile: /etc/ssl/certs/cert-chain-101-mudrac.ffzg.hr.pem
olcTLSCertificateKeyFile: /etc/ssl/private/mudrac.ffzg.hr.gnutls.key

We need to convert OpenSSL key into format which GnuTLS understands:

deenes:/etc/ssl/private# certtool -k < star_ffzg_hr.key > /tmp/star_ffzg_hr.gnutls.key

Than we need to create certificate which includes our certificate and required chain in same file:

deenes:/etc/ldap/slapd.d# cat /etc/ssl/certs/star_ffzg_hr.crt /etc/ssl/certs/DigiCertCA.crt > /etc/ssl/certs/chain-star_ffzg_hr.crt

All is not over yet. OpenLDAP doesn't run under root priviledges, so we have to make sure that it's user is in ssl-cert group and that our certificates have correct permissions:

deenes:/etc/ldap/slapd.d# id openldap
uid=109(openldap) gid=112(openldap) groups=112(openldap),104(ssl-cert)

deenes:/etc/ldap/slapd.d# chgrp ssl-cert \
/etc/ssl/certs/DigiCertCA.crt \
/etc/ssl/certs/star_ffzg_hr.crt \
/etc/ssl/certs/chain-star_ffzg_hr.crt \
/etc/ssl/private/star_ffzg_hr.gnutls.key

deenes:/etc/ldap/slapd.d# chmod 440 \
/etc/ssl/certs/DigiCertCA.crt \
/etc/ssl/certs/star_ffzg_hr.crt \
/etc/ssl/certs/chain-star_ffzg_hr.crt \
/etc/ssl/private/star_ffzg_hr.gnutls.key

deenes:/etc/ldap/slapd.d# ls -al \
/etc/ssl/certs/DigiCertCA.crt \
/etc/ssl/certs/star_ffzg_hr.crt \
/etc/ssl/certs/chain-star_ffzg_hr.crt \
/etc/ssl/private/star_ffzg_hr.gnutls.key
-r--r----- 1 root ssl-cert 3764 Jan 19 09:45 /etc/ssl/certs/chain-star_ffzg_hr.crt
-r--r----- 1 root ssl-cert 1818 Jan 17 16:13 /etc/ssl/certs/DigiCertCA.crt
-r--r----- 1 root ssl-cert 1946 Jan 17 16:13 /etc/ssl/certs/star_ffzg_hr.crt
-r--r----- 1 root ssl-cert 5558 Jan 19 09:23 /etc/ssl/private/star_ffzg_hr.gnutls.key

Finally, we can modify LDAP configuration to use new files:

deenes:/etc/ldap/slapd.d# grep ssl cn\=config.ldif 
olcTLSCACertificateFile: /etc/ssl/certs/DigiCertCA.crt
olcTLSCertificateFile: /etc/ssl/certs/chain-star_ffzg_hr.crt
olcTLSCertificateKeyFile: /etc/ssl/private/star_ffzg_hr.gnutls.key

We are done, restart slapd and enjoy your new certificates!

FSec 2015 - Raspberry PI for all your GPIO needs

September 24, 2015 11:52

When I started playing with Raspberry Pi, I was a novice in electronics (and I should probably note that I'm still one :-).

But since then, I did learn a few things, and along that journey I also figured out that Raspberry Pi is great little device which can be used as 3.3V programmer for AVR, JTAG, SWD or CC111x devices (and probably more).

I collected all my experiences in presentation embedded below which I had pleasure to present at FSec conference this year. I hope you will find this useful.

DORS/CLUC 2015: AVR component tester

June 20, 2015 12:10

Few weeks ago, we had our annual conference DORS/CLUC 2015 on which I had interesting (hopefully) presentation about AVR component tester. Since then, we got video recording of conference, so below you can find embedded presentation and video recording (in Croatian).

Overview of ganeti cluster from command line: ps, kvm, proc and tap

January 30, 2015 12:35

We have been running ganeti cluster in our institution for more than a year now. We did two cycles of machine upgrades during that time, and so far we where very pleased with ability of this cloud platform. However, last week we had a problem with our instances -- two of them got owned and started generating DoS service attack to external resources. From our side it seemed at first like our upstream link is over saturated, and we needed to way to figure out why it is.

At first, it seemed like this would be easy to do. Using dstat,i I found that we are generating over 3 Gb/s traffic every few seconds to outside world. We have 1 Gb/s upstream link, but our bonded interfaces on ganeti nodes can handle 3 Gb/s of traffic, so for a start we where saturating our own link.

But which instance did that? I had to run dstat on every node in our cluster until I found two nodes which had instances which where overloading our link. Using iftop I was able to get hostname and IP address of instances which I wanted to shut down. However, this is where problems started. We didn't have DNS entries for them, and although I had IP and mac address of instances I didn't had easy way to figure our which instance has that mac.

Than I figured out that I can get mac from kvm itself, using ps. Once I found instances it was easy to stop then and examine what happened with them.

But, this got me thinking. Every time I have a troubleshooting problem with ganeti, I basically use more or less same command-line tools to figure out what is going on. But I didn't have a tool which would display me some basic stats about instance, but including mac addresses and network traffic (which in our configuration are tap devices added to bridges). So I wrote, gnt-info which presents nice overview of your instances in ganeti cluster which you can grep to drill-down into particular instance or host.

A tale of false alarm by ConfigServer, CPanel and a hosting provider.

March 06, 2014 14:14

I'm responsible for a couple of CPanel/WHM managed dedicated servers.

We keep them updated, and try to do as little customization as possible outside of what cPanel knows about. We enabled mod_proxy_fcgi and PHP-FPM, so we can use Apache 2.4 MPM Event for our fairly high traffic web site. It's a unfortunate that CPanel doesn't have this configuration available out of the box, but that's for another blog post.

Today early in the morning we got a message from our lfd daemon (a service installed by a free ConfigServer Security & Firewall CPanel plugin installed by our hosting provider):

The following list of files have FAILED the md5sum comparison test. This means that the file has been changed in some way. This could be a result of an OS update or application upgrade. If the change is unexpected it should be investigated:
/usr/bin/ghostscript: FAILED
/usr/bin/gs: FAILED

The funny thing is, nothing upgraded any RPM files in this time window, our /var/log/yum.log didn't mention any upgrades to ghostscript package that provides the /usr/bin/gs binary (/usr/bin/ghostscript is a symlink to gs), we have disabled automatic updates that can be initiated by the cpanel upcp --cron sciprt, but the system us regulagrly kept up to date manually with yum update.

I've reinstalled the package with yum reinstall ghostscript (ghostscript-8.70-19.el6.x86_64 was reinstalled)

and the binary size and md5sum changed like this:

before:
size: 19152 bytes
md5sum: c64b5016d94450b476148c31cfef61ff

after reinstall:
size: 6760 bytes
md5sum: 73db43e258c4b191757b7ba75a883321

This is what actually happened: Our managed hosting provider had apparently changed our setup to upgrade our system packages automatically (probably with best intentions due to recent gnutls issue). And prelinking seems to be enabled on our system, so when upcp (CPanel automatic upgrade cron script that runs periodically) executed /usr/local/cpanel/scripts/rpmup to upgrade system packages, it also did the prelinking step, adding extra prelinking stuff to our /usr/bin/gs binary.

Similar issue described here:

http://linsec.ca/blog/2012/01/23/rpm-v-and-prelinked-binaries/

Dota 2 performance: Linux/native vs Linux/Wine vs Windows 7 on Intel GPU

September 05, 2013 16:35

So how well does the mega popular game Dota 2 work on Linux? I've had some time to make detailed tests on my Intel IvyBridge GPU laptop (Lenovo ThinkPad X230). The graphics settings are the same on all versions.

Dota 2 Windows binary under Wine 1.6
Startup: 35 secs (over ntfs3g userspace fs driver that is not that fast)
Mem: 1.0GB
FPS: 37.5 FPS
Power usage LP mode (patched wine): 34W

Dota 2 Linux native
Startup: 1 min 14 secs (native ext4)
Mem: 2.6 GB
FPS: 40 FPS

Dota 2 Windows native - Windows 7
Startup: 25 secs
Mem: 1.2GB (measured by Windows Task manager)
FPS: 80 FPS
Power usage LP mode: 24W

Test setup:

CPU: Core i5 3320M

Resolution: 1366x768

GPU settings (same on all Dota 2 versions): shadows MEDIUM, textures HIGH, render quality: HIGHEST, all other: OFF, vsync: disabled

GPU settings LP mode (for power measurments above): Shadowd low, effects OFF, textures MED, render quality: LOWEST, fpx_max: 30

Mesa version: git-8b5b5fe (with rendering regression fix from here: https://bugs.freedesktop.org/show_bug.cgi?id=67887)

Linux distro: Ubuntu 13.10, kernel 3.11 drm-intel-nightly, running LXDE

FPS Benchmark method:
in "dota 2 beta/dota/cfg/autoexec.cfg"
cl_showfps 2
playdemo test

For FPS measurement number: look at the last 240 frames when the demo is ending

Memory measuring: RES column with `top`
Startup time measuring: stopwatch until the map is loaded

Analysis of the apitrace trace file:

I've made a trace of Dota 2 with apitrace, revealing possible performance issues.

Before the first frame of the game is drawn 11038 shaders are compiled. That is most likely why the load time is so slow and memory usage is so high. In addition a lot of the shaders being used seem to be recompiled by the Intel driver when rendering frames.

There are 162 frames in the trace I've analyzed, 193 shader recompiles, and 643 different shader programs (each program has 1 VS and 1 FS) used.

In contrast, Wine version of Dota 2 compiles only 220 shaders.

Performance feedback from the Intel driver:

glretrace of apitrace prints driver performance warnings. A sample of some that repeat every frame. These include shader recompile warnings.

575332: glDebugOutputCallback: Medium severity API performance issue 13, Clear color unsupported by fast color clear. Falling back to slow clear.
576094: glDebugOutputCallback: Medium severity API performance issue 14, Failed to fast clear depth due to scissor being enabled. Possible 5% performance win if avoided.
577739: glDebugOutputCallback: Medium severity API performance issue 4, Using a blit copy to avoid stalling on 480b glBufferSubData() to a busy buffer object.
577801: glDebugOutputCallback: Medium severity API performance issue 8, Recompiling vertex shader for program 7901
577801: glDebugOutputCallback: Medium severity API performance issue 9, Didn't find previous compile in the shader cache for debug
577801: glDebugOutputCallback: Medium severity API performance issue 1, Recompiling fragment shader for program 7901
577801: glDebugOutputCallback: Medium severity API performance issue 10, Didn't find previous compile in the shader cache for debug
577801: glDebugOutputCallback: Medium severity API performance issue 3, FS compile took 2.266 ms and stalled the GPU

Warnings in sequence in URL below:
https://gist.github.com/vrodic/6235313

It's interesting that most of this warnings were added by request of Valve back in 2012:
http://lists.freedesktop.org/archives/mesa-dev/2012-August/025288.html

Conclusion:

If you have a memory constrained machine and want to run under Linux, maybe using Wine is a better choice.

I hope Valve cares enough about Linux to fix what they can on their side and work with folks from Intel to fix their performance problems.

You will probably be luckier if you run it on nVidia GPU, since people in general are claiming performance very similar to Windows. Though probably still with slower startup times and higher mem usage.

Some info on how to compile Mesa 32 bit on 64bit Ubuntu:

Unfortunately this doesn't work with old Ubuntu versions, just with 13.10. For old version you must also remove 64 bit versions of the compiler, which was a bit too much of a requirement for me.

apt-get install gcc-multilib g++-multilib binutils-multiarch libx11-dev:i386 libdrm-dev:i386

and as configure complains install others

apt-get build-dep mesa -ai386

wants to install too much and remove some 64 bit stuff I need, and we don't actually need llvm i386 dev stuff for compiling just the Intel driver.

I use this script to compile just the i965 driver for mesa:

sh autogen.sh
make clean
export CFLAGS="-m32 -O3 -mtune=native -march=native -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security"
export CXXFLAGS="-m32 -O3 -mtune=native -march=native -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security"
export PKG_CONFIG_PATH=/usr/lib/i386-linux-gnu/pkgconfig
./configure --disable-egl --enable-glx-tls --with-gallium-drivers= --with-dri-drivers=i965 --enable-32-bit
make -j4

Dota 2 Wine optimization for Intel GPUs

September 04, 2013 16:59

Dota 2 for Linux implements it's 3D engine by using a Direct3D to OpenGL translation layer called ToGL. I assume that this layer can be used in different ways, but for Dota 2 it seems to be used in a less than ideal way as documented previously here. In short, Dota for Linux compiles 11000 shaders on startup, compared to just 220 the Wine version does. This causes much higher memory usage (1.2 GB vs 2.6 GB) and start-up time (35 seconds vs 1:15 min).

With Wine we actually do get the source of their Direct3D to OpenGL layer called wined3d, since Wine is open source. It's funny, the stack used to run Windows version of Dota 2 is actually more open.

Since Dota 2 for Windows when run on Wine actually outperforms native Linux version in some important aspects, and it's framerate is just slightly less, I decided to take a look on improving its performance.

I've used a tool called apitrace to record a trace of a Dota 2 session with wine so I can analyze the OpenGL calls and look at driver performance warnings (INTEL_DEBUG=perf) with qapitrace.

I optimized two things:

1. Reduce the number of vs and ps constants checked

There were many calls to check values of VS (vertex shader) and PS (pixel shader, also called fragment shaders in OpenGL) constants each frame like this:

532550 glGetUniformLocationARB(programObj = 152, name = "vs_c[4095]") = -1

This function is called from shader_glsl_init_vs_uniform_locations() in glsl_shader.c in
wined3d.

It uses GL_MAX_VERTEX_UNIFORM_COMPONENTS_ARB, defined to be 4096 in #define MAX_UNIFORMS in Mesa source.

Dota 2 doesn't need so many uniforms, most checks return -1, and wined3d checks all of values for both VS
and PS uniforms.

I reduced this number to 256, just enough for Dota 2. This saved thousands of calls per frame.

2. Use fast clear depth more often

Intel driver complains about not being able to use fast depth clears because of scissor being enabled. Turns out that device_clear_render_targets() in wined3d device.c doesn't really need to do glScissor for Dota 2, it's probably an optimization that maps better to Direct3D driver.

A small patch including both optimizations is here:
https://gist.github.com/vrodic/6437312

This patch is a hack, and glScissor part probably breaks other apps, so this is just for Dota 2. It maybe could be made in a better way so it could be merged in Wine, but I'm not wined3d expert.

So how faster is it? A solo mid hero on a setup described in the previous blog post used to get 41 FPS. Now it gets 46-49 FPS. Native version is similar to optimized Wine, but in some situations it gets worse than Wine optimized.

Ideas for improvement:

Dota 2 for Linux needs ~7500 calls per frame. Wine version, even after my optimizations needs 37000 (EDIT: just as I was writing this post, there were some improvements, now its about 22000).

There is probably a way to optimize this even more, but it's outside of the scope of an afternoon project, like this was. I'd like to keep on digging though.

Restarting

January 07, 2013 08:28

This week it came to my mind it's been more than a half of a year since my last post. Be assured, it was for a reason. This was probably the most exciting half of a year ever in my life, so let's start from the beginning.

So what changed since then? Long story put short - I got married, I relocated from Croatia to California, and got my first Silicon Valley job. After interviewing and examining my options for a really long time (partly my fault, partly due to a really long interviewing process) with Google and then Facebook, I finally got an offer from Facebook to work in their HQ in Menlo Park, CA as a production/infractructure engineer. The position described was really awesome, basically a great hacking job involving programming, system administration and design, scaling etc. Basically a devops position. I relocated to San Francisco in late September.

Given the opportunity to work on some of the world's largest linux deployments, writing a lot of Python code and debugging some of the hardest engineering problems, coupled with opportunity to work with the top IT talent around and living in geek paradise is really exciting, and I'm sure I'll have much to write about.

Measuring time, pythonic approach

June 07, 2012 19:59

Function wrapping with Python decorators, and code wrapping with Python context managers (i.e. context manager __enter__() and __exit__() functions) can easily be used for measuring time used for running a function or code snippet. Here is the simple implementation of both approaches:

import sys
import time

if sys.platform == 'win32':
  timer = time.clock
else:
  timer = time.time


# use as decorator
def measure_time(func):
  def _measure_time(*args, **kwargs):
    start_time = timer()
    res = func(*args, **kwargs)
    print 'Time elapsed for {0}: {1}'.format(func.__name__, timer() - start_time)
    return res
  return _measure_time


# use as context manager
class MeasureTime:
  def __enter__(self):
    self.start_time = timer()

  def __exit__(self, exception_type, exception_val, exception_traceback):
    print 'Time elapsed: {0}'.format(timer() - self.start_time)

Usage case for both approaches:

>>> @measure_time
... def test():
...     time.sleep(2)
...
>>> test()
Time elapsed for test: 2.00204205513

>>> with MeasureTime():
...     time.sleep(2)
...
Time elapsed: 2.00203204155

Python for Perl programmers #1 - fileinput module

May 07, 2011 15:59

Python for Perl programmers is a set of short posts showing how some of the most common Perl idioms can easily be implemented in Python. This time I'll show you how to translate code chunks using Perl's diamond operator to elegant Python code.

while (<>) {
    ...
}

This loop written in Perl will iterate over all of the lines from standard input, whether it is pipe from another program or the list of the files specified as the program's arguments (argv). The same functionality can be done in Python by using the fileinput module, so the previous code snippet becomes:

import fileinput

for line in fileinput.input():
    ...

The default order of processing is:

sys.argv[1:]
sys.stdin

If file name is specified as "-", the standard input is used instead. You can get more information on fileinput module in official Python documentation.

Managed attributes in Python

October 20, 2010 22:55

Attribute interception is a common pattern that occurs in almost every object oriented language. In addition to only changing the value of some attribute in object, in most cases you will need to implement some additional logic that deals with the changed value. Most straightforward solution for this by using setters and getters. This way the user of the object won't access the attribute directly - he will call the helper functions that hold all the logic needed for fetching and changing the attribute value.

To be honest, there is nothing wrong with this kind of approach, but it just isn't pythonic and elegant. The purpose of this article is to give a brief introduction on more advanced Python topics dealing with this problem. In most cases, you will find them to be just a syntactic sugar around setters and getters, but knowing more about these techniques will help you to read and understand code from more experienced Python programmers, and writing more effective and elegant code of your own.

setters and getters

As we have already mentioned, the most simple approach is to use setter and getter functions. For demonstrative purposes, we will use an example class Person with attribute name that verifies the name is always properly capitalized.

class Person:
    def setName(self, name):
        self._name = " ".join([e.capitalize() for e in name.split()])
    def getName(self):
        return self._name

>>> p = Person()
>>> p.setName("joe smith")
>>> p.getName()
'Joe Smith'

As shown, this just works, but we need to call functions instead using the attribute name directly. The rest of the examples will show how to avoid this kind of programming.

__setattr__/__getattr__and __setattribute__/__getattribute__

Next, we will show how to intercept the access to the attribute, so we can add whatever logic we need to process before the actual access has been made. We can do that by using either the __setattr__ or __setattribute__. Of course, if your goal is to implement something inside getter, and not setter, you would use the __getattr__ and __getattribute__ methods instead. Let's continue with our example - first we will show how to use __setattr__.

class Person:
    def __setattr__(self, name, value):
        if name == "name":
            value = " ".join([e.capitalize() for e in value.split()])
            self.__dict__[name] = value

Note that simply assigning value to the attribute wouldn't work, as it would loop in the __setattr__ call indefinitely, so we must add it to the __dict__ explicitly.

Approach with __setattribute__ is very similar, but there is one notable difference. While __setattr__ and __getattr__ will be called only for those attributes that are undefined, i.e. not on an instance and not inherited from a class, __setattribute__ and __getattribute__ will be called every time you try to access the attribute. In this matter, __setattribute__ and __getattribute__ are more generic as they will be called every time. Again, you need to be careful to avoid loops. While implementing setter with __setattribute__ will be the same as with __setattr__, there is a notable difference while implementing the __getattribute__ method. Here we can't use __dict__[name] as calling it would trigger the __getattribute__ again, causing a loop. To avoid this, you would use the object.__getattribute__ instead of self.__getattribute__.

Properties

Properties allow us to reroute the attribute's set, get or even delete operations to the functions of our choice. To do this we will use the property built-in. The basic syntax that can describe the use of properties is (all of the arguments are defaulting to None, and we will silently ignore the last two arguments for now):

attribute = property(get, set, del, doc)

How to use this? Well, it's pretty straightforward. Our example from previous sections rewritten to use properties looks like this:

class Person(object):
    def setName(self, name):
        self._name = " ".join([e.capitalize() for e in name.split()])
    def getName(self):
        return self._name
    name = property(getName, setName)

Note that we have derived our Person object from object, stating that we want to use is it as the new style class. This is required for properties to function normally (of course, if you're using Python 3.0+ you won't need this as new style classes are implied).

Properties can also be coded with decorators. In this case, base decorator @property is used for getter, and name.setter is used for setter. Again, the following example is merely syntactic sugar for the previous example:

class Person(object):
    @property
    def name(self):
        return self._name
    @name.setter
    def name(self, name):
        self._name = " ".join([e.capitalize() for e in name.split()])

In contrast with the previous methods, properties manage only a single attribute, and therefore are easier to write as you don't need to remember all the gotchas of __getattr__, __getattribute__ etc.

Descriptors

The final method we will examine are descriptors. In their essence, descriptors are superset of properties, and property built-in we have used before is only a specific case of descriptors. Descriptors allow us to reroute attribute's get and set (or even delete) operations to methods of another class.

The template code for coding descriptors is:

class Descriptor:
    def __get__(self, instance, owner):
        ...
    def __set__(self, instance, value):
        ...
    def __delete__(self, instance):
        ...

We will again ignore attribute deletion for now. A class containing any of these three methods is descriptor. Omitting the __get__ or the __delete__ method will cause that particular type of access as unsupported, however omitting the __set__ method will not render the attribute as read only, it will only allow its name to be redefined (and therefore descriptor will be hidden). If you want to make your attribute read only, you should implement __set__ with pass statement or raise an exception on assignment.

Argument names for descriptor methods are somewhat strange. Let's start with __get__ method. The owner argument is the name of the class to which the descriptor instance is attached, and the instance argument is the instance of the object through which the attribute was accessed. Meaning of the instance argument in the __set__ method is the same, and the value argument holds the value to be assigned to the attribute.

How do we use a descriptor? We just assign it to the attribute name. Our example, this time with descriptors, looks like this:

class Descriptor(object):
    def __get__(self, instance, owner):
        return instance._name
    def __set__(self, instance, value):
        instance._name = " ".join([e.capitalize() for e in value.split()])
 
class Person(object):
    name = Descriptor()

Again, we must use new style classes if we're not using Python newer or equal to Python 3.0.

An interesting thing with descriptors is that they can hold state. OK, technically speaking every of the previously described methods can also hold state, at least in the form of global variables, but by using descriptors you can use them elegantly in their own namespace - the descriptor class and instance.

Conclusion

This is by far not a complete tutorial on using any of techniques described here, as there are many more things to be described, but they are simply out of the scope of this article. You should read the official Python documentation or some of the better books on Python programming (I would strongly recommend Learning Python, 4th ed. from O'Reilly) to get the complete overview of managed attributes possibilities.

Budget Surfer 0.0.1

August 29, 2010 15:34

Jučer sam htio istražiti koliko je našeg zajedničkog novca utrošeno na softverske licence za MS Windowse, Office i ostale proizvode više ili manje lako zamjenjive sa FOSS ekvivalentima. Jedine informacije koje sam zasad uspio dobiti su u dokumentu "Poseban dio Državnog proračuna Repulike Hrvatske za 2010. godinu i projekcije za 2011. i 2012." koji se nalazi ovdje http://mfin.hr/hr/drzavni-proracun-2010 .

Predpostavljam da je i Vizualizacija proračuna od projekta vjetrenjača napravljena iz istog podatkovnog izvora.

Kako bih produktivno gubio vrijeme odlučio sam napraviti mali program koji će importirati ovaj Excel u SQL bazu i omogućiti lakše "surfanje", filtriranje itd.

Ako se predpostavi da je softver u stavkama "INFORMATIZACIJA*", u podstavkama "Rashodi za nabavu neproizvedene imovine", onda ukupni iznos iznosi oko 24.5 milijuna kuna.


Informatizacija, Rashodi za nabavu neproizvedene imovine

No zanimljivo, misteriozne podstavke s naslovom "Rashodi za nabavu neproizvedene imovine" (koje nemaju nikad detaljnije podstavke) ne pojavljuju se samo u stavkama "INFORMATIZACIJA*", nego i npr. "RAČUNALNO KOMUNIKACIJSKA INFRASTRUKTURA U VISOKIM UČILIŠTIMA I JAVNIM INSTITUTIMA" od 14 milijuna kn i mnoge druge kategorije što nas dovodi do ukupnih oko 105 milijuna kuna.

Rashodi za nabavu neproizvedene imovine

Ono za što je predpostaviti da ima neku vrijednost u nabavkama softvera u Informatizaciji su stavke

"Nematerijalna proizvedena imovina" za koje je predpostaviti da se radi o softveru rađenom po narudžbi, tj koji se proizvodi za potrebe rada države. Ukupni iznos ovoga je oko 74 milijuna kuna.

Softver po narudžbi

Moj kupus, ŠBBKBB iliti TODO lista

August 24, 2010 23:00

Ako niste voljni čitati nadobudne i vjerovatno pretenciozne osobne rantove i brain dumpove slobodno preskočite ovaj post.

Pošto izgleda da je danas dan kada dajem otkaz još jednoj u nizu closed source (i closed mind dodao bih) firmi, možda je najbolje da se opustim uz nekoliko naivnih i praktičnih ideja za koje znam kako ih izvesti. Vjerovatno ću šarati od ideja za pojednostavljenje državne birokracije preko ideja za Android aplikacije, do bojnog plana za osvajanje javnog sektora od Linux desktopa u Hrvata.

No prvo o tome što mi je činiti u sljedećem razdoblju: Trebam naći poslodavca koji će shvaćati i prihvaćati da su mi briga za obitelj, prijatelje, zajednicu i moje sebične interese (fakultet recimo) na prvom mjestu. Vjerujem da takav već postoji, a ako netko ima ideju ili prijedlog slobodno se javite.

Ajmo polako s idejama:

Down to earth:

1. Proširiti postojeću Kostovu inicijativu za Android Market account tako da razvijemo i održavamo nekakav HULK-ov sistem donacija za domaće Free Software Android / Whatever aplikacije.

2. Android aplikacija koja skenira barcodove prehrambenih proizvoda i ima offline bazu sa crnom listom Ebrojeva u proizvodu. Samo uzmete proizvod u dućanu u ruke, skenirate mu barcod i dobijete crveno ili zeleno ovisno o tome ima li proizvod štetnih E dodataka (naravno postoji opcija za pregled detalja E brojeva a ako smo online i drugih detalja vezanih uz proizvode). Dodatno bi se ta radnja mogla iskoristiti da se proizvod zabilježi kao kupljen, te tako možemo imati i neki pregled / povijest potrošnje, a i jednostavniji način izbora namirnice kod brojanje kalorija ili neko drugo sofisticiranije praćenje prehrane. Naravno za organski proizvedene proizvode bez bar koda treba smisliti nešto drugo za jednostavno identificiranje (brzi izbor voće, povrće, boja, oblik, last recently used etc)

Blue sky:

Ideje u domeni odnosa države i građana:

I. Zdravstvena knjižica:

Kako relativno često mijenjam poslove, primjetio sam jednu glupu ostavštinu prošlih vremena: Posebnu zdravstvenu knjižicu koju iz nekog čudnog razloga treba mijenjati svaki put kada se promjeni radni status, a valjda i svaki put kada završite osnovnu, srednju ili fakultet. Ta zdravstvena knjižica ionako sadrži datum isteka koji ne znači baš ništa (jer osiguranje formalno može "isteći" i prije - o tome kako se uopće u socijalnoj državi može dogoditi da nemate osnovno zdravstveno osiguranje malo poslje). Dakle jedino što bi na njoj trebalo biti bitno je broj zdravstvenog osiguranja, odnosno OIB odnosno JMBG odnosno broj osobne izkaznice odnosno broj putovnice. Sve su to u biti ključevi za identifikaciju građana RH, i u teoriji bi bilo dovoljno imati recimo samo osobnu ili putovnicu (sa barcodom ili npr. qr codeom za strojno čitanje tog malog ključa). Status zdravtvenog osiguranja bi se transparentno u pozadini promjenio kada bi dobili prvi posao, dali otkaz, završili fakultet itd itd. Ionako ga uvijek zapravo imamo (neki imaju i bonus dopunsko ako su bez posla ili slično) i jedino što je bitno je da se to na nekog "knjiži" radi poreznih razloga, ali to stvarno ne znači da svaki put moramo mijenjati tu plastiku. Predpostavljam da fino zarađuje na toj plastici, a ako nije u pitanju neka korupcija, opet me fascinira glupost u javnom sektoru (nadam se da je samo glupost, jer tada je valjda lakše popraviti ovo).

A drugi problem/pizdarija oko zdravstvenog osiguranja je to što se može dogoditi da ostanete neosigurani jer ste recimo mislili da datum isteka koji piše na zdravstvenoj kartici zapravo znači datum isteka zdravstvenog osiguranja. Ne, to znači jedino i samo to koliko vrijedi taj komad plastike. U idealnom slučaju status zdravstvenog osiguranja bi se računalno provjerio prilikom svakog korištenja neke zdravstvene usluge, i gdje bi se korisnik samo upozorio da njegov status nije jasan (recimo zbog završetka studentskih prava gdje je uobičajeno da zdravstveno osiguranje sa roditelja/fakulteta "prelazi" na zavod za zapošljavanje ili budućeg poslodavca).

II. Još malo o zdravstvu:

Bilo bi lijepo kada bi laboratoriji za pretrage odnosno specijalisti automatizmom digitalno slali informacije o nalazima liječniku opće prakse, odnosno nadležnom specijalistu. Vjerujte, ovo bi uštedjelo kilometre i kilometre koji ljudi sami prelaze kako bi raznosili nalaze okolo liječnicima. U tom slučaju bi i liječnici mogli nazvati pacijente čim nalazi stignu, a ne da pacijenti moraju raditi upite svako malo. Dakle u računalnom žargonu, "digital event based a ne analog poll bazed event handling". Bilo bi lijepo i da pacijenti građani) imaju token za pristup centralnoj birokraciji gdje mogu sami pogledati svoj zdravstveni karton, rezultate pretraga, plan liječenja, dijetu, prisjetiti se ocjena u školi ili za one za nekom vrstom alzeihmerove bolesti, vidjeti koliko nekretnina imaju, tko su im roditelji, gdje su rođeni, itd itd :) I tako, ima tu puno etičkih i drugih pitanja koje treba razmotriti prije deploymenta. No predpostavljam da ne želimo da ovako nešto jednom za nas radi neki closed source sistem ili nedaj bože Facebook :)

U varijanti gdje JMBG ili OIB ili broj osobne ili broj putovnice stvarno postaju ono što jesu, a to su ključevi s kojim se identificira građanin-pojedinac, nije nam više potrebna niti radna knjižica niti pokaz niti Xica, a u idealnom slučaju (kada bi imali taj "token").

Od domaćih sustava, možda je upravo sustav iza Xice (ili studomata) baza za ovakvo nešto.

III. Stavimo Linux na desktope javnog i državnog sektora.

Stvarno, želimo li i danas, uz solidan i uoptrebljiv desktop na linuxu plaćati svake godine milijune za MS licence? Možda možemo otvoriti malu firmu koja bi se bavila supportom i specifičnim potrebama korisnika u državnom sektoru za novce utrošene na MS licence i time bar malo spriječiti odlijev novca odnosno nepovoljan omjer uvoza i izvoza?

IV. Pomognimo Marku Rakaru da http://proracun.pollitika.com/2010/korisnici.html ima više informacija, real time informacija, podataka o aktivnim natječajima u javnoj nabavi i slično ili nagovorimo državu/vladu/sabor da ovakave projekte kvalitetno odrađuju Narodne novine tj da ih financira sama država.

V. Pripremimo sve ove projekte tako da lako postanu dio nečeg većeg kada uđemo u EU ili se ujedinimo sa USA (kiberkomunist approved). Koristimo postojeća (Free software) riješenja gdje god možemo. Nemojmo se bojati ulagati u globalnu zajedničku infrastrukturu.

VI. Podržimo Zajedničko i zadružno stanovanje http://bit.ly/bEbcwK, Reciklirano imanje - Vukomerić http://bit.ly/arRK0U i slične projekte.

VII. Nacionalizirajmo ponovo Hrvatske Telekomunikacije ili barem:

Definirajmo jasnije javno pravo na DTK. Prisilimo HT na članstvo u CIX-u. Dajmo im dozvolu da postavljaju optiku, stavimo cijenu za najam niti na 100 kn po mjesecu po korisniku kako bi se brže vraćala investicija, ali svakako omogućimo da infrastrukturu od prvog dana mogu koristiti i alternativci. Regulirajmo maksimalne i minimalne cijene. Postavimo u zakon da svaka telefonska parica / veza može prenijeti neki normalni broadband a ne trenutnih 128K kako bi prisilili monopolistu da izgradi lokalnu infrastrukturu (ili stvorimo budžet za ovo dodatnim oporezivanjem operatera sa veliki tržišnim udjelom). Borimo se za simetrični broadband (sa korisnim upload brzinama). Borimo se za low latency broadband. Borimo se za 100megabitne, i gigabitne internet veze. Zadnje tri stavke omogućavaju i vrlo dobar peer to peer cloud - što nam treba za nekakav Diaspora-like social networking). Iskoristimo stvarno podatkovne kapacitete javnih poduzeća. Kad planiramo novu infrastrukturu za nova ili postojeća naselja, predvidimo i prostor sve bitno (da i optički internet pristup), kako nebi nepotrebno prekopavali kvart/naselje svako malo.

VIII. Dolazimo na razmjene vještina.

IX. Stvorimo računala koja možemo uzgajati kao što uzgajamo biljke i koja si stvaraju energiju sama kao što ih stvaraju biljke i koja imaju mrežu korjenja za povezivanje slično kao što i biljke imaju korijenje (prvi korak, pogledajte http://bbf.openwetware.org/).

X. Razno.

Diskusija o ovom blog postu nalazi se i na Friendfeed-u

Back on the road

July 22, 2010 20:20

After almost a year of pause, I've decided to revive this blog. Much has changed in the meantime, so I'm full of ideas and themes to write about. Some of them are:

cloud computing: I've almost completely abandoned all of the applications I've been using before (mail client, feed reader, office applications, personal information management applications etc), and switched to online and cloud versions (primarily Google's)
Android platform: more like continuation of the previous point - I can now access all of the cloud services I use from literally anywhere, thanks to this great mobile platform
development: migration from subversion to git, several themes on Perl, Python and shell scripting
systems administration: several capacity planing, storage optimizations and general system tweaking themes

So, stay tuned! :)

Making Android development more enjoyable

July 04, 2010 11:25

Here's some things Google should look into:

0. Include android market in the emulator/test environment

That should be fast and easy to do.

1. Try finding some ways to speed up the compile/test cycle:

a) Avoid moving too much stuff arround

Currently every time an .apk file must be produced (compressed) on the developers machine, containing all the code and application resources. This apk that contains everything is then transferred to the phone or emu. It's then unpacked on the phone and the install procedure is run. It doesn't matter if you just changed one class file, the whole thing is moved around.

I understand that permissions are managed during the install process, but this could be solved in some other way like trusting the code for that app by default.

b) Make the test environment run in native code during the development process

I know that using an emulator was a nice and fast solution for you, but a lot of time and energy is lost on emulating ARM. Davlik runs on x86, and all other code also. Why don't we just make a jailed/chrooted native environment for testing available? I know that this is not a straightforward thing to implement on Windows, and there might be difficulties making this run on Mac OSX, but it's really worth it.

2. Improve the visual layout editor.

There's so many ways this could be improved, but making it faster/more intuitive is the general idea.

One quick suggestion: Try making it easier to jump to respective code implementations or have an option to generate event handlers etc. code if no code referencing that UI object is found.

Example of a great code editor/UI designer integration is Borland Delphi. Even really old versions have ease of use that Android developers could only dream of. Android API is more abstract and UIs have relative layouts, but most of the great concepts from Delphi still apply.

Item 1 b) could also create possibilities for the UI designer tool.

HTC Desire prvi dojmovi

May 30, 2010 17:19

EDIT: 26.5.2010: par novih detalja, + vesela informacija da je Samsung I9000 Galaxy S koliko vidim HTC Desire killer.

Kupih na pretplatu u VIP-u. Onu najmanju od 30kn, + 10 kn za radijsku frekvenciju + još

oko 10 kn za iskorištavanje "besplatnih" minuta + po želji oko 30 kn za 200 MB prometa. Dakle obvezao sam se na trošenje minimalno 50tak kuna mjesečno sljedeće dvije godine.

+ 2400 kn za sam telefon i nije toliko loš deal.

Kad sam kupovao HTC Desire znao sam da nije savršen, da ekran nije najbolje čitljiv na suncu, da uskoro dolaze novi uređaji temeljeni na bržem 45 nm Qualcomm Snapdragonu (moguće da će biti 1.5 GHz dual core!) i da će vjerovatno HTC kilaviti sa update-om na Android 2.2 za Desire , ali sam znao i da ima trenutno najbrži procesor, najviše memorije i odličnu kvalitetu zvuka, što mi je jako važno.

Opasno me zainteresirala Motorola Milestone (Droid u USA), koja ima tipkovnicu, nešto sporoji procesor, ali zato dosta brži GPU. No ona je pala na audio testu na gsmarena.com, što provjeram za svaki mobitel koji uzimam u obzir pošto sam se opekao sa Nokiom N79 gdje mi je glazba bila neslušljiva od distorzija. Ta Nokia je bila blizu savršenstva, tj. Nokia E52 bi bila još bliža, ali sam odlučio zamjeniti sada prilično zrelu Symbian platformu sa Android-om koji je ipak njubi...

~~Još nema kvalitetne offline navigacije (Symbian Nokie imaju~~ Garmin XT koji je odličan). iGo ima aplikaciju Motonav,(ima i potpuniji My way, ali neznam dali postoji verzija za Androide sa velikom rezolucijom), detalji ovdje. Navteq karte su bolje od Teleatlas karti izgleda. Garmin XT je još uvijek bolji, lijepše renderira karte i brži je.

Izgleda da je potrebno koristiti nekakav Task manager za ubijanje aplikacija, koje u pravilu nemaju exit ili quit opciju i vole sjediti u backgroundu.

Android issue 3708, Akcelerometar ne može dojavljivati podatke kada je ekran ugašen, tj. kad je mobitel u sleep mode-u. Ovime aplikacije koje recimo mjere broj koraka postaju skoro pa beskorisne jer puno brže popiju bateriju. Drugi primjer je navodno odlični Smart alarm. A i ja planiram napraviti aplikaciju koja inteligentno koristi akcelerometar u pozadini.

Još nisam naletio na dobar DivX player. HTC je nešto obećao u kasnije SW updateu. Ali poznato je da se ne može vjerovati ovakvim obećanjima od proizvođača mobitela.

Opera 5 Mini je još uvijek brzinom navigacije (back, forward, history) bolja od Android browsera. Komentar dolje spominje i Dolphin, ali to nisam probao. Android stock browser naravno bolje podržava JavaScript i HTML5 od Opere 5 mini.

root-anje Androida (za stavljanje custom ROM-ova) još nije stabilno za HTC Desire, neki ljudi uspiju brickati telefone kod flashanja, neki ne (meni osobno ovo nije toliko bitno, iako me živcira što se neke meni nepotrebne HTC aplikacije, Stocks recimo, stalno vraćaju u listu aktivnih procesa, iako ih nisam dozvao)

Još nema custom ROM-a koji je stabilan, tj implementira sve HW mogućnosti uređaja

Hardver je dovoljno različit od Nexus One-a da se na njega ne može staviti Nexus One ROM, što će vjerovatno značiti da će HTC Desire korisnici dobiti Android 2.2 kasnije od Nexus One korisnika (nadajmo se da griješim)

Aplikacije se po defaultu ne mogu instalirati na SD karticu, što ograničava prostor za instalaciju, HTC Desire ima samo 147 MB za instalaciju aplikacija. Ovo riješava Apps2SD (treba root), ili Android 2.2, ali oboje je trenutno nedostupno za Desire

Poneki bug u browseru

3D je definitivno sporiji od iPhone-a 3GS, Motorole Droid, a vjerovatno i iPhone-a 3G

FM radio softver je dosta neintuitivan, i zna se skršiti tako da je potreban reboot mobitela da bi se radoi ponovo osposobio, dok uredno nastavi trošiti 100% CPU-a, što je smrt za bateriju

Aplikacije koje se plaćaju nisu dostupne kod nas?

Aplikacije koje još tražim:

Music player koji ima file browser i playa po redu iz trenutnog foldera.

Bonus

Aplikacije koje sam instalirao, redom kojim sam ih instalirao (- neke igre):

Google Sky Map, Compass (Snaptic), ConnectBot, ASTRO File manager, Accelerometer Values, Android Sensor Data, Sensor List, Input Benchmark, Ultimate Stopwatch, Spare Parts, PowerTutor, fring (Skype, SIP, odlično), GPS Status, GPS Test, GPS Essentials, SID Player, Robo Defense FREE, Task Manager (Rhythm Software), android-vnc-viewer, Android System Info, NetCounter, Neocore (benchmark za grafiku), NenaMark1 (još jedan GPU bench), Tuner - gStrings, Robotic Guitarist Free, Splash Piano (prilično loš klavir, ali nema bolji), Buddy Runner, Endomondo Sports Tracker, SportPal OS1.5, Hi-Hiker Pro, Pedometer, Cardio Trainer (da, testiram personal GPS enabled aplikacije za trčanje/hodanje/bicikliranje, Cardio Trainer je simpa), Android SPL METER, KeepScreen, Fractoid, 2 Ballz Physics Toz, System Panel App / Task Manager, Free Advanced Task Manger, Spectral Audio Analyzer, FxCamera, PicSay, Ethereal Dialpad (synthezer), Camera Illusion, RTA Audio Analyzer, Barcode Scanner, Shazam, Google Goggles (malo nestabilno), ColorFlashlight Fun Flashlight, Call History (xiaoheng.liu), Mobile Metronome, Paint Joy, Raging Thunder LITE, Open GPS Tracker, WiFi Analyzer, Google Translete, OpenGL Demos, Sipdroid VoIP, Linpack for Android

Photo by Goran Zec

TwinLab Calcium 1000 sa Vitaminom D SMRDI

May 26, 2010 17:11

Ovo je otvoreno pismo poslano gosp. Ivanu Tudoru iz firme Agram 89 d.o.o, koja je zastupnik tvrtke TwinLab.

Nedavno sam u jednoj ljekarni u Dugom Selu kupio proizvod firme koju zastupate, TwinLab Calcium 1000 sa Vitaminom D.

U ljekarni gdje sam kupio proizvod su me uputili na vaš broj 01 3888 555. Vaše kontakt informacije dobio sam jer su me na vas uputili kada sam nazvao 01 6138 000.

Računao sam na pomoć u ljekarni oko odabira najboljeg dodatka u prehrani koji sadrži kalcij i Vitamin D i bio mi je preporučen ovaj proizvod.

Moj otac je nedavno operiran zbog rane faze adenokarcinoma (odstranjen mu je dio plućnog krila). Nedavno sam naišao na studiju koja govori da kombinacija Vitamina D i kalcija pomaže kod uobičajenih vrsta karcinoma (http://www.ncbi.nlm.nih.gov/pubmed/17556697).

Iako ovo istraživanje nije provedeno specifično za adenokarcinom, mislio sam da ne može škoditi ako ocu kupim dodatak u prehrani koji sadrži Vitamin D i kalcij.

Otvorio sam novo kupljeni proizvod i nakon što sam uzeo jednu tabletu primjetio sam u popisu sastojaka Titanijum dioksid, koji se u ovom proizvodu vjerovatno koristi zato da bi tablete bile bijele boje. Titanijum dioksid, E171 (http://www.ukfoodguide.net/e171.htm). Na priloženoj internetskoj stranici piše da je TiO2 zabranjen u Njemačkoj i da se može detektirati u limfnim čvorovima i plućnim stanicama nakon ulaska u ljudsko tijelo.

Pošto TiO2 nema biološku ulogu u ljudskom tijelu, a spominje se da je iritant i da je potencijalno kancerogen, pitam se zašto se ovakvi proizvodi uopće prodaju, tj. zašto se u proizvod stavlja potencijalno opasan dodatak koji nema nikakvu funkcionalnu ulogu. Mislim da marketing i igranje sa psihologijom nebi nikad smjeli imati prednost nad funkcijom posebno kada se radi o ljudskom zdravlju i kada dodatak koji se stavlja iz marketinških potreba može štetiti.

Stranica wikipedije http://en.wikipedia.org/wiki/Titanium_dioxide sadrži ukupno 6 referenci na studije kojim su okarakterizirani negativni učinci na ljudsko zdravlje.

Nisam zadovoljan sa ovim proizvodom i neću ga koristiti, a niti moj otac. Proizvod sam platio 173 kn i imam račun iz ljekarne. Nije mi iznimno važno da dobijem natrag novac i nekakvu "zadovoljštinu", nego da se ljudi informiraju. Povrat novca bio bi lijepa gesta sa vaše strane.

Možda ću pokrenuti inicijativu o zabrani proizvoda koji sadrže ne funkcionalne, kozmetičke E dodatke a koji imaju uz sebe povezane negativne efekte (E110, E171 itd). Molim zainteresirane da mi se pridruže.

Pokrenuo sam raspravu o ovoj temi na adresi:

http://ff.im/c2BgG

No siguran sam da se o ovome raspravlja i na mnogim drugim mjestima na internetu.

The end

May 01, 2010 13:43

I'm closing this blog until further, since I do not have much time to invest in writing coherent, meaningful posts. Longer, in-depth texts will from now on appear on my homepage at zvrba.net Commenting is disabled since a lot of spam has lately appeared.

Finally!

November 29, 2009 18:54

On thursday, 2009-11-26, I have finally defended my PhD! I'll be starting on the new job on tuesday...

Unpredictability of execution times

November 12, 2009 12:10

I have written an article on intrinsic randomness of modern CPUs that you can fetch here. Originally, I tried to publish it in LWN, but since I haven't heard from their editors in over a week, I'm publishing it this way.

Windows 7 hype

October 24, 2009 14:31

Sometimes it's really interesting to see what good marketing can do for a mediocre or even good product, even if the product itself doesn't offer anything revolutionary.. These days all you can find on RSS, IT news portals, Twitter etc can be summarized into one term - "Windows 7". First of all, I must admit - although I'm a long time Linux user/admin and that's the main thing I do for living, I got my hands on Windows 7 RTM. So here are my two cents on the new Microsoft's product and hype around it after using it for a little longer than a month.

Undoubtedly, Windows 7 is the most advanced desktop OS ever released from Microsoft. I wouldn't say it is the best desktop OS currently available, but it is probably the most consistent one. As I've never used Windows Vista, it was quite a surprise to log into this new OS, and to find dozens of new features. On the first look, it is far more visually appealing than XP. Much of concepts that I've been missing in the previous versions of Windows are now here, for example media and document libraries, better network manager, network profiles (home, domain, work, public etc). Advanced users (like myself :)), will be happy to hear that PowerShell 2.0 is included in the OS by default, and you event get a simple IDE for it (PowerShell ISE). As I can hear, it even runs much faster and uses much less resources than Vista, but I can't confirm this as I've never used Vista.

Of course, there's no such thing as perfect software product, and some things must be criticized. First of all, 10 GB for basic OS installation is simply silly. With some other operating systems (most notably Linux), you get OS and almost all other software you need in much less, like 3-4 gigs at most. Memory usage is also pretty high for an OS kernel and several core services, and I don't accept "RAM is cheap nowadays" as an argument. OS should use much less memory resources than it is the case with Windows 7. Period. I've also noticed a constant disk activity even when I'm away from my computer, and this is really annoying if your hard disk is not among the most silent ones. Finally, there is a nasty bug I've encountered (and some of my colleagues, and thousands of people as I can see on the Internet) - problem with crashing after locking the screen (either with win+L or after an idle period) while being logged on domain network. I've temporary "fixed" this problem with disabling screen locking in the registry, but there is still no fix from Microsoft. My colleagues have even reported this problem to Premier support, but we are still waiting for the solution.

But let's return to the main thing that made me start writing this article, marketing and hype about the Windows 7. Microsoft announces features like Aero shake, snap and peek as revolutionary, and that's what makes me laugh. First of all, you could have used such features years before Windows 7 on Linux (Beryl and Compiz). And seconds, silly GUI effects like those previously mentioned can hardly be described as something revolutionary (they could have at least "invent" virtual desktops... but no). In my opinion Mac OS or Linux are far ahead of Windows if you're main goal is to have nice desktop with eye candies. I mean, it's nice to have such things, but you can live without them.

To summarize, if you're a long time Windows user, you should switch to Windows 7. If you're Linux of MacOS user, you should probably stick to your OS. Windows 7 is polished, OS somewhat faster, and with some new features, but it's not nearly so "unique and revolutionary" as Microsoft announces it.

A tricky bug

October 17, 2009 16:10

Today I debugged a piece of code that looked like this:

printf("foobar\n");
if(fork() == 0)
  exit(0);

When the program is run from the terminal, the output is as expected: a single line containing "foobar". However, when the output is sent to a pipe or redirected to a file, two lines appear in the output. (Actually, the problem was a bit more complicated: there were n fork() calls and there were exactly n additional copies of the output). After a bit of digging around and not finding any obvious fault, I asked about it on IRC.

It turned out that stdout is fully buffered when it is attached to a pipe or a file. What happened is that fork() duplicated also the internal I/O buffers, which got flushed after the child process exited, thus producing extra output. I solved the problem by inserting a fflush(NULL); statement before forking, which flushes buffers of all output streams.

Foxit PDF reader -- no go

September 03, 2009 23:10

Today I tested the Foxit PDF reader, version 3.1, since many people recommend it as a faster and lighter-weight alternative to Acrobat Reader. Well, its performance is disappointing: opening a 3MB PDF with Foxit takes 2-3 seconds before the first page is displayed. Opening the same PDF with Acrobat Reader (version 9.1) displays the document with no delay at all. I attempted to open few other PDFs with Foxit, with the same result: annoying pause before I could view the document, but with Acrobat Reader showing the document immediately.

So long Foxit, Acrobat Reader is still my favorite PDF reader.