2012-01-26

Using NVIDIA Quadro 10000M (Lenovo W520) with Kubuntu 11.10

Unfortunately, NVIDIA Quadro 10000M doesn't work on Lenovo W520 with Kubuntu 11.10 from the out of box. Here is how I set up.

We need to do three things:
  1. pci=noacpi in the kernel option (see the my last blog section 5 boot set up.) If you have still problem try acpi=off, but in this case I have only one CPU core detected. See https://wiki.ubuntu.com/DebuggingACPI
  2. Set graphics device to discrete graphics since Nvidia Linux driver seems have not yet Optimus support.
  3. Update the Nvidia driver to 290.10. (280.13 seems not work on this machine. The error is:  udev-fallback-graphics main process ($PID) terminated with status 1)
I assume you are still in Optimus or Integrated chip mode. In this mode, you can have X running, so it is a good start.
  • I first updated the Linux kernel to 3.0.0.15.
There are two ways: Ubuntu way and direct installation.  Ubuntu-way should be later better, but in my case, glxinfo does not work.  And the fowllowing error I got.
% glxinfoname of display: :0.0X Error of failed request:  BadWindow (invalid Window parameter)  Major opcode of failed request:  137 (NV-GLX)  Minor opcode of failed request:  4 ()  ...
  • Anyway, here is the Ubuntu way for the record. (Currently, this doesn't work, but I hope later this works.)
  • 
    
      
    sudo apt-get purge nvidia-current
    sudo add-apt-repository ppa:ubuntu-x-swat/x-updates
    sudo apt-get update
    sudo apt-get install nvidia-current
    sudo apt-get install nvidia-setting
    
    
    Now you have the driver version 290.10.
cd /etc/X11
sudo cp xorg.conf xorg.conf.mybackup
sudo nvidia-xconfig
Note: Section "Device"'s BusID should be "PCI:1:0:0" in my case. The old version of nvidia-xconfig say "PCI:0:2:0" and can not found the driver. Unfortunately, this doesn't work (2011-1-27). glxinfo casts an error.
  • The following is not the Ubuntu way, but, this worked
  1. Download NVIDIA-Linux-x86_64-290.10.run from http://www.nvidia.com/content/DriverDownload-March2009/confirmation.php?url=/XFree86/Linux-x86_64/290.10/NVIDIA-Linux-x86_64-290.10.run&lang=us&type=GeForce
  2. Get the console!
  3. sudo apt-get purge nvidia*
  4. sudo sh ./NVIDIA-Linux-x86_64-290.10.run (I don't install the 32bit compatible OpenGL libs. That might cause a problem. Others I answer yes.)
  • Reboot and pray.
Set Discrete Graphics on. Nvidia Linux driver seems have not yet supported Optimus.
  • Press F1 on boot time, Config -- Display -- Graphics Device to Discrete Grap. Save and boot.
At this point I can see the 1920x1080 resolution screen and NVIDIA X Server settings works (This is nvidia-settings-update 280.13 one.)

I hope you can also enjoy the Nvidia card on Lenovo W520.

2012-01-25

Dualboot with WDE (Whole Disk Encryption) Windows 7 and Linux Kubuntu 11.10


See the great reference http://www.iceflatline.com/2009/09/how-to-dual-boot-Windows-7-and-linux-using-bcdedit/

Introduction

I have PGP-WDE on a Windows 7 laptop, but I want to have a Linux dual boot partition and also with whole disk encryption. This is how it was done.

Detailed situation
Usually,
  • A virtual machine is convenient.
  • Home directory encryption is enough.
  • Just use PGP-WDE is possible.
But, sometimes life is not so easy. If you are in the following situation, you have the same problem with me.
Then, we need another solution.

Suggested solution
  • Windows 7 partition is encrypted by PGP-WDE.
  • Linux (Kubuntu 11.10) partition is encrypted by dm-crypt.
  • (My machine is Lenovo W520 with 500GB HD.)


How to do it?

1. Decrypt PGP disk

Use PGP Disk tool to decrypt your disk. Because you can not install Linux on a decrypted disk.  It took 8 hours for 500GB disk on Lenovo W520.

2. Partition the disk
  • Use Windows Disk Management or any other tool to shrink the Windows 7 partition.
  • Note: Windows 7 has a small extra partition (around 300MB size). This is necessary and do not delete it.
3. Install Linux (Kubuntu 11.10)

Push ThinkVantage button, and push F12 to boot other drive. I use Kbuntu 11.10 and boot via USB key.

Install Kubuntu
  • The alternate CD contains whole disk encryption menu. Kubuntu Desktop only offers home directory encryption option. (You also need a Desktop CD later)
Partition disk

  • We will have "/boot", "/", and swap partition. swap may not needed. 
  • Note: /boot should not be encripted. 
  • Choose Manual. I made following partitions 
  • primary 200MB ext4 /boot, /dev/sda4 (You should remember /boot partition's device name) 
  • logical 241GB K lvm 
  • logical 8G swap
  • I choose the / partition for lvm 
Configure encrypted volumes
  • Create encrypted volumes 
  • Choose /dev/sda6 (lvm), /dev/sda5 (swap) 
  • Encryption method: Device mapper (dm-crypt) 
  • Encryption: aes 
  • Finished encryption 
  • Input pass phrase 
Now it starts to install, but without asking where the boot loader is installed. (I was scared since I have already break Windows 7 when PGP-WDE encryption is active and master boot record is overwritten. I try to recover MBR, but, I could not fix it and need to re-install Windows 7.)

After install the base system, the installer asked us where the GRUB boot loader should be installed.
  • Say 'No' to: Install the GRUB boot loader to the master boot record? 
  • Install the GRUB boot loader in the /boot linux partition (my case, /dev/sda4) 
  • Reboot the Linux (from USB key or Live CD, not Alternate CD)4.


Dual boot set up from Windows 7 partition
  • After the reboot from Desktop CD/USB key, we will set up the dual boot.
  • Copy the relevant info to a USB key (mounted on /media/myusbkey in this example) or similar:
 sudo dd if=/dev/sda4 of=/media/myusbkey/linux.bin bs=512 count=1
  • This makes only 512 byte file.
  • Boot Windows 7.
  • Copy linux.bin from the USB key to some place on the Windows partition, e.g. C:\linux.bin
  • Run the command line tool as administrator (right click the icon for that).
  • Run the following commands:
bcdedit /create /d "Linux" /application BOOTSECTOR
  • "Linux" can of course be something else. The command outputs a long ID, use that instead of {ID} in the following commands.
bcdedit /set {ID} device partition=c:
bcdedit /set {ID} path \linux.bin
bcdedit /displayorder {ID} /addlast
bcdedit /timeout 10
5. Linux boot set up
  • I failed boot in normal mode, but I can boot recovery mode.
  • It seems normal mode make the encryption passphrase input prompt invisible. It looks like a hung up. But this seems Linux is asking disk's pass phrase.  Just we can not see the prompt.
  • I changed the grub settings. Don't do quiet mode and graphical mode. I could not find /boot/grub/menu.lst file in Kubuntu 11.10. There is a /etc/default/grub file.
  1. boot with recovery mode (You will be asked the passphrase.)
  2. sudo vi /etc/default/grub
  3. Edit the line: GRUB_CMDLINE_LINUX_DEFAULT="splash nomodeset pci=noacpi" from "quiet splash"
  4. run sudo update-grub (I think this changes /boot/grub/grub.cfg)

  • Note: If you don't have pci=noacpi here, Lenovo W520 with Linux kernel 3.0.0.15 hung up when using NVIDIA card. If you have still problem, try acpi=off. (Intel integrated graphics card has no problem.)

6. Windows 7 PGP partition encryption
  • Use the PGP tool to re-encrypt the Windows partition (not the whole disk). 240GB encripytion took 6 and half hours in my case.
  • When you want to encrypt Windows partition, you must choose: Encrypt Whole Disk, then, the PGP Desktop asked you to create new user. I choose Windows password and default settings.
7. Finished
Now your whole harddisk is encrypted except linux /boot partition. But, there is no secret here.


Thanks to Daniel, Joachim, and Joerg for many hints.

2012-01-16

Passing a user defined python object to C++ code(4)


How to access to self.face_idx_list. extracting a numpy.int64 type failed.

The difference between this article and the last one is numpy.array contains numpy.int64 instead of numpy.float64. We can access numpy.float64 value as the following:

vec[i] = boost::python::extract< float >(    float32_3_obj.attr("__getitem__")(i));

I.e., we can convert numpy.float64 value to float value by boost::python::extract< float >(f). However, when we try to convert numpy.int64 to int as the following code, it fails.

vec[i] = boost::python::extract< int >(
    int32_3_obj.attr("__getitem__")(i));

We can not convert numpy.int64 by boost::python::extract< int >(i). We will get the following error:
TypeError: No registered converter was able to produce a C++ rvalue of type int from this Python object of type numpy.int64.
That's strange. float is OK, but int isn't.  I looked up the web, there are some articles regarding with this issue. http://boost.2283326.n4.nabble.com/extracting-a-numpy-int32-type-td2701573.html I understand as that: numpy registered a conversion function from numpy.float64 to float, but, there is no conversion function registered numpy.int64 in 64bit machine. There was a suggestion to use python function int() to convert the numpy.int64 values. This works in the python world, but not in the C++ world.

I suggest the following code:
int vec[] = {0, 0, 0};
for(int i = 0; i < 3; ++i){
    object int64_obj = int32_3_obj.attr("__getitem__")(i);
    vec[i] = boost::python::extract< int >(
        int64_obj.attr("__int__")());
}
This code first gets a numpy.int64 object as a python object, then uses numpy.int64's attribute method __int__ that converts numpy.int64 to int.

This gives you information about the conversion numpy.int64 to int.  But some of readers might wonder how to find this information. How to find this information seems more useful, actually it is easy, ask python.

The former article mentioned that all the python members are attributes, and the attributes can be seen themselves a sequence of the attributes using dir() function. Also we can use type() function to see the type. In this case, from the python interpreter, we can use type() to get the type information.

>>> import numpy>>> a = numpy.array([1,2,3])>>> type(a[0])
Now we see the numpy.array element is 'numpy.int64' instead of 'int'. This type object is again python object, we can use dir() to get the attribute information. Then you can find an attribute called __int__. Let's use that attribute and see the type of it:
>>> type(a[0].__int__())
Now we know this is the conversion function we are looking for Therefore, we can use this conversion attribute method to get the int object. When we have int object, then we can use extract< int > to get a C++ int.
This is the basic of how to access to a user defined python object from C++ code.

I hope this article is useful to you.

Passing a user defined python object to C++ code(3)


How to access to the self.vertex_list.

This time I will explain how to access to the self.vertex_list of TriMesh. This is a list of length three numpy.array. We have already seen how to get the python list (dict's keylist) in my former article http://shitohichiumaya.blogspot.com/2010/08/boostpython-how-to-pass-python-object.html.

How to access to the float numpy.array element is the following.

void print_float32_3(boost::python::object const & float32_3_obj)
{
    // check sequence length is 3 or not.
    if(boost::python::len(float32_3_obj) != 3){
        std::string const objstr =
            boost::python::extract< std::string >(
                boost::python::str(float32_3_obj));
        std::cerr << "print_float32_3: arg is not a float[3] obj ["
                  << objstr << "]" << std::endl;
        return;
    }


    float vec[] = {0.0f, 0.0f, 0.0f};
    for(int i = 0; i < 3; ++i){
        vec[i] = boost::python::extract< float >(
            float32_3_obj.attr("__getitem__")(i));
    }
    std::cout << "float[3] = " << vec[0] << " " << vec[1] << " "
              << vec[2] << std::endl;
}

First, numpy.array is a sequence, therefore, we can ask length of it by boost::python::len(). We check the length is 3 or not since I defined it is three dimensional vector in the python code. Then we can access to each element through the attribute __getitem__. The python code puts float values to numpy.array, thus we can extract float value with extract< float >. Strictly speaking, numpy.array contains numpy.float64 instead of float, but, extract< float > still works here.

The last member of TriMesh, the triangle face index, should be done in the same way. Actually I would like to say that, but, there is a glitch here in numpy. We can not access this elements in the same way. We will get an errer:
TypeError: No registered converter was able to produce a C++ rvalue of type int from this Python object of type numpy.int64(). 
I would like to talk about this issue in the next blog.

Passing a user defined python object to C++ code(2)


The python code of the last article contains a simple triangle mesh (TriMesh) and sets some data to the object. The mesh has only two triangles that forms a rectangle. It's not so exciting scene, but it is sufficient as an example. The scene is shown in the following image. A camera looks up the rectangle.

Figure 1. TriMesh scene example (one rectangle)

This article explains how to access to the three members of TriMesh. Each member of a python object is an attribute, there is a way to access to any attribute in general, but I think it is complicated. I assume the programmer has an knowledge about a specific python class. For example, I know TriMesh object has three member variables
(three attributes).

  • self.__material_name: a string, but this is a private member. We should access to this member via get_material_name() method.
  • self.vertex_list: A list of numpy.array, that contains length three numpy.float64.
  • self.face_idx_list: A list of numpy.array, that contains length three numpy.int64.
How to access self.__material_name.

A TriMesh object is passing as pytrimesh in the code.
std::string const matname =
   extract< std::string >(pytrimesh.attr("get_material_name")());
We know the name of the attribute. Therefore, we can use boost::python::object's method attr to get the method function, then call the operator() to get string object. Please note the () of the end of the line. It is not
pytrimesh.attr("get_material_name")
but, it is
pytrimesh.attr("get_material_name")().
The operator() of boost::python::object returns an boost::python::object, we need to use extract<>() to extract a std::string. This is actually relatively simple since the contents is a build-in object of python.

I would like to stress two points here.

  • If we access to non-existing attribute, an AttributeError exception is casted. The following is an example:
    boost::python::object const vlist = pytrimesh.attr("vertex_list_non_exist");
    
  •  We can not access to an private member variables (private attribute). Therefore, the following code fails.
     std::string const matname = extract< std::string >(pytrimesh.attr("__material_name"));
I will show how to pass a list of numpy.array to C++ code in the next article.

2012-01-13

Passing a user defined python object to C++ code(1) Source code


Once I explained how to pass a python build-in object (dict, list, and
string) to C++ code using python.boost.
http://shitohichiumaya.blogspot.com/2010/08/boostpython-how-to-pass-python-object.html

This time, I will explain about how to pass a user defined python object
to C++ via python.boost. I will also explain why numpy.array's
numpy.int64 object casts an error when I do extract< int >. This casts a
TypeError (TypeError: No registered converter was able to produce a C++
rvalue of type int from this Python object of type numpy.int64()). Then
I will show you how to solve this problem.

Here is an python example code.
-----
#
# test pass object 2 module, python side implementation
# Copyright (C) 2012 Hitoshi
#
import numpy
import passobj2_mod

class TriMesh(object):
    """TriMesh: simplified triangle mesh primitive for python.boost
    demo"""

    def __init__(self, _mat_name):
        """default constructor."""
        self.__material_name = _mat_name
        self.vertex_list       = []
        self.face_idx_list     = []

    def get_material_name(self):
        """get material name
        \return material name"""
        return self.__material_name

    def info_summary(self):
        """summary information
        \return summary information string
        """
        ret_str =\
            '# materialname = ' + self.__material_name         + '\n' +\
            '# vertices     = ' + str(len(self.vertex_list))   + '\n' +\
            '# faces        = ' + str(len(self.face_idx_list)) + '\n'
        return ret_str


tmesh = TriMesh('diffuse_material')

# vertices
vlist = []
vlist.append(numpy.array([552.8, 0.0,   0.0]))
vlist.append(numpy.array([0.0,   0.0,   0.0]))
vlist.append(numpy.array([0.0,   0.0, 559.2]))
vlist.append(numpy.array([549.6, 0.0, 559.2]))
# faces (obj file like)
flist = []
flist.append(numpy.array([1, 2, 3]))
flist.append(numpy.array([1, 3, 4]))

tmesh.vertex_list   = vlist
tmesh.face_idx_list = flist
print 'triangle mesh info\n' + tmesh.info_summary()

pobj2 = passobj2_mod.passobj2()
pobj2.pass_trimesh(tmesh)
-----

This code defines a simple triangle mesh class. Each triangle vertex is
represented by numpy.array that have three floats (a three-dimensional
vector). Each triangle face is represented by three vertex indices.  You
may notice the index started with 1. Because I often use the wavefront
obj file format for a geometry file. In that file format, the index
started with 1. This TriMesh has a material name as a private member.

The C++ python.boost code that accesses to python TriMesh object is the
following.

-----
/// How to pass the python dict to the C++ function/method.
///
/// Copyright (C) 2010-2012 Hitoshi
///
/// test for using C++ class from python: pass an object that contains
/// list of numpy object
///
/// trimesh is an object, that has vertex list and face list.

#include <boost/python.hpp>
#include <boost/python/object.hpp>
#include <boost/python/extract.hpp>
#include <boost/python/list.hpp>
#include <boost/python/dict.hpp>
#include <boost/python/str.hpp>

#include <stdexcept>
#include <iostream>

/// using namespace only for example
using namespace boost::python;

//----------------------------------------------------------------------
/// print one numpy.array (actually length 3 sequence), type float
///
/// \param[in] float32_3_obj float[3] object
void print_float32_3(boost::python::object const & float32_3_obj)
{
    // check sequence length is 3 or not.
    if(boost::python::len(float32_3_obj) != 3){
        std::string const objstr =
            boost::python::extract< std::string >(
                boost::python::str(float32_3_obj));
        std::cerr << "print_float32_3: arg is not a float[3] obj ["
                  << objstr << "]" << std::endl;
        return;
    }

    float vec[] = {0.0f, 0.0f, 0.0f};
    for(int i = 0; i < 3; ++i){
        vec[i] = boost::python::extract< float >(
            float32_3_obj.attr("__getitem__")(i));
    }
    std::cout << "float[3] = " << vec[0] << " " << vec[1] << " "
              << vec[2] << std::endl;
}

//----------------------------------------------------------------------
/// print one numpy.array (actually length 3 sequence), type int
///
/// \param[in] int32_3_obj float[3] object
void print_int32_3(boost::python::object const & int32_3_obj)
{
    // check sequence length is 3 or not.
    if(boost::python::len(int32_3_obj) != 3){
        std::string const objstr =
            boost::python::extract< std::string >(
                boost::python::str(int32_3_obj));
        std::cerr << "print_int32_3: arg is not a int[3] obj ["
                  << objstr << "]" << std::endl;
        return;
    }

    int vec[] = {0, 0, 0};
    for(int i = 0; i < 3; ++i){
        // MARK_03
        object int64_obj = int32_3_obj.attr("__getitem__")(i);
        vec[i] = boost::python::extract< int >(
            int64_obj.attr("__int__")());
    }
    std::cout << "int[3] = " << vec[0] << " " << vec[1] << " "
              << vec[2] << std::endl;
}

//----------------------------------------------------------------------
/// accessing numpy.array list
///
/// \param[in] numpy_list_obj list of numpy.array.
/// \param[in] is_float       true when float array
void access_numpy_array_list(object const & numpy_list_obj,
                             bool is_float)
{
    int const len = boost::python::len(numpy_list_obj);
    std::cout << "len(numpy_list_obj) = " << len << std::endl;
    list cpp_list = extract< list >(numpy_list_obj);
    for(int i = 0; i < len; ++i){
        // operator[] returns python::boost::object.
        object one_vec = cpp_list[i];
        if(is_float){
            print_float32_3(one_vec);
        }
        else{
            print_int32_3(one_vec);
        }
    }
}

//----------------------------------------------------------------------
/// C++ object which python can instantiate
class PassObj2 {
public:
    /// constructor
    PassObj2()
    {
        // empty
    }

    /// pass a python object
    /// \param[in] pytrimesh a trimesh object
    void pass_trimesh(object const & pytrimesh) const {
        // access to material name via the access method. MARK_01
        std::string const matname =
            extract< std::string >(pytrimesh.attr("get_material_name")());
        std::cout << "trimesh.get_material_name() = " << matname << std::endl;

        // access to python list
        object const vlist_obj = pytrimesh.attr("vertex_list");
        object const flist_obj = pytrimesh.attr("face_idx_list");

        std::cout << "--- vertex_list ---"   << std::endl;
        access_numpy_array_list(vlist_obj, true);
        std::cout << "--- face_idx_list ---" << std::endl;
        access_numpy_array_list(flist_obj, false);
    }

private:
};

/// importing module name is 'passobj_mod'
BOOST_PYTHON_MODULE(passobj2_mod)
{
    class_("passobj2")
        .def("pass_trimesh",
             &PassObj2::pass_trimesh,
             "pass python trimesh object to c++ method")
        ;
}
\end{verbatim}

The following code is a build script. Of course, you need python.boost
installed in your system.
\begin{verbatim}
#! /bin/sh -x
# Copyright (C) 2012  Hitoshi
#
PYTHON_INCLUDE=`python-config --includes`
MOD_CPP_SOURCE_BASE=passobj2_mod
g++ ${PYTHON_INCLUDE} -DPIC -shared -fPIC ${MOD_CPP_SOURCE_BASE}.cpp -o ${MOD_CPP_SOURCE_BASE}.so -lboost_python
\end{verbatim}

If you run this program, you will see the following result.
\begin{verbatim}
@[34]bash % python test_passobj2_mod.py
triangle mesh info
# materialname = diffuse_material
# vertices     = 4
# faces        = 2

trimesh.get_material_name() = diffuse_material
--- vertex_list ---
len(numpy_list_obj) = 4
float[3] = 552.8 0 0
float[3] = 0 0 0
float[3] = 0 0 559.2
float[3] = 549.6 0 559.2
--- face_idx_list ---
len(numpy_list_obj) = 2
int[3] = 1 2 3
int[3] = 1 3 4
---

The C++ output shows the same as in the python code.

This time I showed the code. Let's talk about the details in the
following blog entries.

2012-01-07

Optimist





2012-01-03

A relationship between eigenvalues and determinant


This year started with heartbreak. The multiplication of all the  eigenvalues is determinant of the matrix. (and the sum of all the  eigenvalues is trace of the matrix. These properties are amazing!)  I read an astonishing simple proof using diagonalization of a matrix. This blog is about that proof.

Matrix A can be diagonalize to
Actually, the way to reach this is a bit long and the proof I will show you here is based on this. But, I hope you could agree this. The determinant of this relationship is

By the way,

Because,

A determinant is a scalar value, therefore, they are commutative. Therefore,
What a simple proof this is! But, one thing we need care: this is only true when $S^{-1}$ exists. So, this is not a complete proof, but, I like this simple proof.

References

  • Gilbert Strang, Introduction to Linear Algebra, Chapter 6.