Wednesday, April 24, 2013

Some notes on find and replace using Vim, rename, and perl in command line...


Using rename in bash script

As mentioned in the previous post [1], I need to replace dots to other tokens in the filenames of a bunch of eps figures. The following script was used to do the work [2]:

#!/bin/sh
for f in $(find . -name "*.eps" -type f)
do
    echo "found: "$f
    # option -n is useful to preview the renamed results:
    rename -v 's/(\d+)\.(\d+)\.(\d+)/$1-$2-$3/' $f
done

In this case, I learned how to keep some parts of the old string and to replace other parts. The key concept was to use parentheses to group the parts we want to keep and then use $n to indicate the nth group in substitution expression. Use ``91.1.8.eps'' as the example:
  • \d stands for digits
  • \d+ means at least ONE digit
  • (\d+) hold the parts which are respectively 91, 1, and 8 in this example
  • $1 corresponds to the first group which is 91
  • $2 corresponds to the second group which is 1
  • $3 corresponds to the third group which is 8
Therefore, the dots between the digits will be replaced by dashes.

Substitution in Vim

After renaming all the eps files, I had another more complicated problem. All the corresponding filename strings resided in the tex files also had to be changed! At the beginning I edited one of the tex file in Vim and played with the substitution command in it. The final command I used was [3][4]:

:%s/\(\d*\)\.\(\d*\)\.\(\d*\)/\1-\2-\3/gc

Note that there are some minor differences when writing the expressions. Some of the modifications, e.g. the escaping backslash, were due to the difference between BRE and ERE [5].

Find and Replace in multiple files


Although I could do the find and replace works in Vim, it was not a good idea when there were maybe hundreds of such files. To write a bash script was my first thought and with information found on the internet [6][7][8] I got a usable script as the follows:

#!/bin/sh
for f in $(find . -name "*.tex" -type f)
do
    echo "found: "$f
    perl -p -i -e 's/(\d+)\.(\d+)\.(\d+)/$1-$2-$3/' $f
done

---
[1] XeTeX -- using dots in eps filenames would cause errors
[2] batch renaming with the rename command

[3] Vim Regular Expressions 101: Grouping and Backreferences
[4] Search/Replace in Vim

[5] Basic Regular Expressions and Extended Regular Expressions
[6] Eeasy Search and Replace in Multiple Files on Linux Command Line
[7] bash find directories
[8] Find file or directory in whole directory structure



Monday, April 15, 2013

Ubuntu 12.04 doesn't print PDF file via network printer

My office PC has been linked to two printers via network (intranet I think...) and had been worked fine until recently. Although I have installed and used Ubuntu 12.04 for a while but have not printed files often. Several weeks ago I tried to print some documents but the printers just gave me strange error messages and stopped working. Today I tried to print something again and had the same problem with the printers. This time I decided to make it work.

At the beginning I had only vague keywords and got no useful searching results in return. I tried to launch LibreOffice Writer to create a simple test file and it was printed successfully, but after saved as PDF the printing was failed. Then I noticed that the documents which had failed in printing were also all PDF files. So the problem could be the file type.

I used PDF as one of the searching keywords and found some bug reports of Ubuntu. I followed some suggestions in one of the threads [1] and got little success.

The approach of updating cups-filters with precise-proposed didn't work to me [2]. Actually I didn't see any update packages after I enabling the precise-proposed option.

The working one was changing settings of the printers via command line [3]. I made one of the printer worked with PDF documents by using settings as the following:
$ lpadmin -p Hewlett-Packard-HP-LaserJet-P3005 -o pdftops-renderer-default=gs
$ lpadmin -p Hewlett-Packard-HP-LaserJet-P3005 -o pdftops-max-image-resolution-default=0
where Hewlett-Packard-HP-LaserJet-P3005 is the printer name. I deleted the second setting by using
$ lpadmin -p Hewlett-Packard-HP-LaserJet-P3005 -R pdftops-max-image-resolution-default
and the HP printer still worked fine when printing PDF documents.

The other Xerox printer, however, still didn't work after changing the settings.


---
[1] Printing on PostScript printers (or printers with PostScript-based driver) not working
[2] #20 of the above thread
[3] #17 of the above thread

Thursday, April 11, 2013

XeTeX -- using dots in eps filenames would cause errors

I had a set of tex files which included many eps figures and were compiled successfully by using the latex+dvips+ps2pdf commands. But due to some Unicode issues I've shifted to XeTeX for at least several months [1]. A strange problem, however, emerged when I was invoking xelatex to compile the same set of tex files. The error message was:
! Unable to load picture or PDF file './EPS_FILE_DIR/91.1.8.eps'.
That's strange because I remembered that I have compiled other tex files with eps figures with xelatex flawlessly. I found the successfully compiled files to make sure it still could be compiled in my machine. It did. So the problem might be caused by the 91.1.8.eps itself. Suddenly it occurred to me that maybe the dots confused the xelatex command, so I changed its filename to 91_1_8.eps and solved the problem.

Although the problem has been solved, I still have no idea about why the dots could cause such a problem. I also tested with a 91.1.8.jpg file and to my surprise it passed the compilation without the error message.

I don't know whether other files (such as png, bmp, pdf, ...) also have similar problems, but I decided not to use dots to name my files anymore.

---
[1] XeTeX -- using the system fonts for CJK tex file

Monday, April 08, 2013

Some problems caused by Vistalizator

In the previous post I said using Vistalizator to change the display language in Win7 is easily. It was easy indeed, but there were some problems and I have only solved one of it.


Thursday, March 28, 2013

Rename the bolg title

My original blog title was ``人 teh 飛, 天 teh 看'' in Taiwanese and the meaning was ``human being are flying and God is watching.'' It came from my old hobby: RC airplanes and helicopters.

After graduating from the university and leaving the lab, my work has been nothing to do with aerial vehicles. I have had no time to keep the RC flying hobby and gradually my interesting (partly due to my work) has been shifted to programming for vision applications.

Maybe one day I will have time to fly RC aerial vehicles again, but for now I am focusing on the topics of image processing and programming and hoping I could get more experiences in the future. So I changed the blog title, which is simple and contains no non-ascii characters, and with the tiny change I could help myself to focus on techniques I want to develop.

Monday, March 25, 2013

Test Kinect in Ubuntu 12.04

In the previous post I tried to make the Kinect (for Windows) work in my Win7 with x86 driver. Actually my first attempt was to try to make the Kinect work in my Ubuntu 12.04 but failed, so I switched back to Win7 to make sure the Kinect is workable.

Several days ago I wanted to test the Kinect in my Ubuntu 12.04, and of course I googled and got OpenKinect. I followed the installation procedure but got no luck and before proceeding on I decided to make sure the Kinect is okay (and therefore `the previous post' about the test in Win7).

This time I followed another tutorial by igorbarbosa and finally got the Kinect working in my Ubuntu 12.04.

My installation steps were slightly different from that described in igorbarbosa's tutorial. I believe it is due to the version changes of avin2's SensorKinect driver.

Now I am sharing my installation and hope it might be useful to you.

Friday, March 22, 2013

Vistalizator -- Easy way to change display language in Win7

Warning: Do NOT use Vistalizator to change your display language. I have encountered something annoying and decided to restore my system. I will post some screenshots of the applications in my Win7 which were affected by  Vistalizator.

For the problems cause by Vistalizator in my Win7, see this post.

Test Kinect in Win7 x64 using x86 driver

In the previous posts [1][2] I tried to rebuild a VS project using Kinect given by my colleague and encountered some problems. Because my Win7 is the x64 version but the project had been built for x86 (Win32) platform, the rebuilding work in my Win7 machine cost me and my colleague almost a whole working day (without my colleague's help, I may give up the rebuilding).

Although I changed the project configuration to x64 [2], there were still problems with other libraries, for instance, OpenCV, built along with the project for x86 platform. When I was wondering the next step, my colleague suggested me to try the x86 driver of Kinect and if it works, just keep sticking on the Win32 setting of the project configuration.

Hmm, it seemed I had taken a long route and headed back to the origin... XD

Okay, the happy news is the x86 driver really worked in the x64 Win7, and here is a working log of the test.

Thursday, March 21, 2013

Change VS2005 project from Win32 to x64

I was rebuilding a Win32 project in my x64 Win7 PC. The online sources [1][2] told me just to change the active project configuration from Win32 to x64, but my VS2005 has no x64 option. Then it occurred to me that maybe I didn't install something related to x64 modules when installing the VS2005, so I checked it by using the installation package.

Wednesday, March 20, 2013

'WIN32': No such file or directory

I was rebuilding a project given by my colleague and encountered strange error message:

c1xx : fatal error C1083: Cannot open source file: 'WIN32': No such file or directory

This error was caused by an empty path variable [1] $(OPEN_NI_INCLUDE) which (I guess) indicates the directory containing the installed OpenNI SDK. It was empty because I forgot to restart Visual Studio after install the SDK... :-p

So the solution is simple: restart Visual Studio and the path variable will be set.

---
Edit:

After solving the previous problem, I got another similar problem which related to the Linker path.

I used the set command [2] in cmd of Windows to the list environment variables about OpenNI:
OPEN_NI_BIN64=C:\Program Files\OpenNI\Bin64
OPEN_NI_INCLUDE=C:\Program Files\OpenNI\Include
OPEN_NI_INSTALL_PATH64=C:\Program Files\OpenNI\
OPEN_NI_LIB64=C:\Program Files\OpenNI\Lib64
Then I changed $(OPEN_NI_LIB) to $(OPEN_NI_LIB64) and the new error message said the file OpenNI.lib cannot be found, and the reason was it should be  OpenNI64.lib in my system. The right place to change the library file name is in the following path:
Properties -> Configuration Properties -> Linker -> Input -> Additional Dependencies

---
Ref:
[1] Cannot open source file: 'WIN32': No such file or directory
[2] How can I display the contents of an environment variable from the command prompt in Windows 7?

Friday, March 01, 2013

[PCVP] Gaussian blurring and a minor imshow problem

As mentioned in the previous post (To-Do List of 2013), I am trying to learn something by following the examples listed in Programming Computer Vision with Python (PCVP).

I've done some exercises without any systematic or consistent record. So I think it's time to post some results here to record my own understanding as well as to push myself to keep going forward.

The exercise here is a test of Gaussian blurring given in pp.31-32 of the PCVP book. The following is my code and output images.


from PIL import Image
from PIL import Image
import numpy as np
from scipy.ndimage import filters
import matplotlib.pylab as plt 

img = Image.open('../data/empire.jpg')
im0 = np.array(img) #original image
im1 = np.array(img.convert('L')) #convert to grayscale
im2 = filters.gaussian_filter(im0,2)
im3 = filters.gaussian_filter(im0,5)
im4 = filters.gaussian_filter(im0,10)

for i in range(5):
    plt.subplot(1,5,i+1).set_axis_off()
    plt.title( '(' + str(unichr(97+i)) + ')' )
    #plt.imshow(eval('im'+str(i))) # This will output the grayscale image with colors 
    plt.imshow(eval('im'+str(i)), cmap=plt.cm.Greys_r)

plt.show()


From left to right: (a) original image, (b) grayscale image, (c)  Gaussian filter with σ = 2, (d) σ = 5, (e) σ = 10.

Actually, the Gaussian blurring has no problems but the convert('L') and the imshow() functions bothered me for a while. I expected the imshow() function will gave me a grayscale image as the one shown in the PCVP book, but it gave me an image with colors as the second one listed in the following figure.


The solution I found [1] and tested is to add a cmap option to tell the imshow() which colormap [2] I want it to apply on the output image.

---
Ref:
[1] Display image as grayscale using matplotlib
[2] Matplotlib Color Maps

Wednesday, January 23, 2013

iBus 的行列輸入法 (Array30 input method of iBus)

If you want to use Array30 input method in iBus, remember to choose ``ibus-array'' package instead of ``ibus-table-array30''. The former one acts identical to that runs on Windows and makes fast input possible.

Once you installed ibus-array successfully, there will be an icon with blue character as the following one:
ibus-array icon

If your icon is a red one looks like this:
ibus-table-array30 icon
then you have the wrong choice.


Wednesday, January 16, 2013

To-Do List of 2013

Hmm... It's already 16th January, and suddenly I feel like to make a to-do list of this year. Actually, I've never made any annual plan before, but I think it's would be good to list what I want to do in the year on my own blog and to keep tracking my progress of the to-do items.

The following list shows things I am doing or want to do:
  1. to take open courses on Coursera
  2. to complete EGGN 512 Computer Vision course on YouTube
  3. to read and to do exercises of Programming Computer Vision with Python

Coursera is a great place to find something to learn. And I think joining a course to keep learning progress is a good idea. So far I've taken two courses and found they are fun: Introduction to Astronomy and Programming Languages.

For the EGGN 512 Computer Vision by William Hoff, I've followed the videos in last year but was failed to keep learning. So this is my second trial to follow the course at my own tempo. I don't want to fail it again.

It is not only the videos. There is a reference book Computer Vision: Algorithms and Applications by Richard Szeliski. This could be a heavy burden for me to read it thoroughly.

Finally, I would like to pick up Python programming again. It has been out of my life for a while and I want it back. Because I have to learn these stuff in limited time and energy, I decided to learn computer vision techniques with Python as the programming tool.

In fact my list could be longer, but the presented items will definitely consume out my leisure time. So I think it's better not be too greedy or I might get nothing as the end of 2013.

Okay. List has been determined.

Get thing done. And just do it. :-)

Tuesday, October 16, 2012

Make XeTeX show quotation marks properly

Because of some annoying CJK font problems, I've switched from traditional LaTeX to Unicode-capable XeTeX.

After several trials, I finally made it worked on my computer and got the result which fitted almost my expectation. I said ``almost'' because there was a little problem of the quotation marks. In stead of showing expected curly marks, XeTeX outputted the marks in original text mode.

The solution I found is to add a bit of setting as follows.
\defaultfontfeatures{Mapping=tex-text}
\setmainfont{YOUR_FONT_NAME}

or simply
\setmainfont[Mapping=tex-text]{YOUR_FONT_NAME}
Actually I have no idea about the difference between the two implements listed above. My own guess is: the first one provides global effect while the second one affects only on the main font?

---
ref:
http://tug.org/pipermail/xetex/2008-October/010929.html

Monday, October 15, 2012

CThruView Transparent Image Viewer -- test in Ubuntu 12.04

In addition to qiv mentioned in the previous post, I found another image viewer which could do some amazing things when viewing transparent images. It is CThruView written in Java.

However, my trial got some problems. The transparent background was replaced by a gray one, and its visibility can only be reduced along with the foreground image by setting the opacity of overall image. The following screenshot shows my test.

At the first trial, I though the fails might be due to the OpenJDK which is the default installation in Ubuntu. So I found the method to install Sun Java 6 but got the same result.

Another failed test was the option ``Click-Thru'' which had no function. Except for these, other options worked properly.

Sunday, October 14, 2012

qiv -- Viewing image with transparent background

In Ubuntu, there is a image viewer named qiv which can show image files with transparent backgrounds.

For installation, just type
$ sudo apt-get install qiv

Invoke qiv in command line, and there are many many options to meet most requirements.

To show the image with its transparent background, the argument -p has to be used:
$ qiv -p IMAGE_FILE.png

Below is a screenshot of my own test. Note that the original image was save as png file with transparent background.

Use qiv to show image with transparent background.

Thursday, October 04, 2012

Visual Studio 2005: the symbol xxx is not defined

I've encountered the problem several times. Every time I have to use Ctrl+F to go through all the source code and it's somewhat annoying.

The only solution I could think of is to delete the *.ncb file or rebuild the project. And it's what I found on internet when I launched the search.

Unfortunately, it didn't work for me.

Fortunately, I finally got a solution which save my live.

Here is the method to solve the ``symbol xxx is not defined'' error:

Edit -> Intellisense -> List Members


XeTeX -- using the system fonts for CJK tex files

After using LaTeX for several years, I found XeTeX might be the best choice for Unicode files and therefore for the production of CJK documents.

When producing the CJK documents, it will be convenient to make the system fonts available for XeTeX. Here is a simple method to make it work.

To install system fonts, put *.ttc or *.ttf  files to the following folder:
/usr/share/fonts/truetype

Then build the font information cache files:
$ sudo fc-cache -f -v 

---
ref:
Mac下的Latex中文解決方案: XeTeX
http://www.hitripod.com/blog/2011/04/xetex-chinese-font-cjk-latex/
http://www.ubuntu-tw.org/modules/newbb/viewtopic.php?post_id=230060#forumpost230060
http://www.tmes.mlc.edu.tw/wiki/index.php/Freefonts
http://www.ptt.cc/bbs/LaTeX/M.1315925855.A.C38.html

Saturday, March 03, 2012

Set Vim in Windows to work with Unicode

My Windows has been installed and set for traditional Chinese environment and occasionally it makes some trouble when I need text files encoded in Unicode instead of Big5.

I've started to write some personal working logs in Markdown syntax with Vim. It's simple and quick. But when I tried to compile the plain text into HTML file, the Chinese characters introduced trouble for the Python implementation of Markdown. The error message was something like:
UnicodeDecodeError: 'utf8' codec can't decode byte
To prevent this kind of error, I thought the best approach was to make Vim work with Unicode in my Windows. The settings were added in C:\Program Files\Vim\_vimrc and they are as follows:

set fileencoding=utf8
set fileencodings=ucs-bom,utf-8,default,latin1

The first line tells Vim to save the text in utf8 and the second line gives Vim to open and convert the files in the given order of encoding.

Sunday, January 08, 2012

Cannot set Chromium as your default browser?

I was using FreeMind to manage information of my work, and there were several links embedded in the mind map. When I was trying to open the links, they were opened in Firefox instead of Chromium that has been my default browser. I tried to set Chromium as the default browser in the Preferences tab but it said ``Chromium cannot determine or set the default browser.''
The solution was found on Ubuntu Forums. In the thread, user cavalier911 provided a method using update-alternatives to set the default browser. The command is as follows.

$ sudo update-alternatives --config x-www-browser

Pretty easy and works like a charm.