Embedded Linux – Memory Management Line by line [1]– Intro to MMU

For embedded system, memory management varies from system to system, as you have your custom address space and custom peripheral devices and memory chips. Sometime, the porting of kernel may fail if the memory management is not carefully handled.

Years ago, uclinux was well-known as a linux operating system without MMU, which is more suitable for embedded systems back at that time. As FPGA has become more and more powerful, embedded OS can support MMU now. And for Linux running on Xilinx FPGA with microblaze or PPC, the MMU is required, as said in the documentation.

In this blog series, I am going to tell the story about memory management of Linux OS line by line. The story begins by introducing the MMU of microblaze.

MMU of Microblaze on Xilinx FPGA\

Memory Management Unit (MMU) is a piece of hardware, which bonded closely with CPU (microblaze/PPC here) to provide memory management functionality for softwares, especially operating systems.

MMU can function in two modes:

1. Real Mode: effective address are used to access physical memory directly.

2. Virtual Mode: effective address are translated into physical address to access physical memory by the virtual-memory management hardware in microblaze.

The mode is controlled by the 18th bit (VM) of rmsr (Machine Status Register), as shown in fig. 1.


                                 fig. 1 Machine Status Register


At the very beginning, the MMU runs in real mode, as the reset value of VM is 0. After OS properly set the first several entry in TLB (Translation Look-aside Buffer), the MMU will switch into Virtual Mode.

In virtual mode, whenever an address is requested, it goes through the following procedure:


To understand this procedure, we need to know so-called Translation Look-aside Buffer (TLB). TLB can be viewed as a cache for address translation. The translation between virtual address and physical address in virtual mode is done via a kind of mechanism called 2-level page table translation. In microblaze, the translation goes as follows:


There are certain addresses that are frequently accessed, the translation is stored in TLB, the same we do for data cache. If the requested data is not found in TLB, a TLB miss (similar as cache miss) is generated, followed by the page table translation and TLB update operation.

There are four registers used for accessing TLB from software: TLBLO, TLBHI, TLBX, and TLBSX.

TLBLO: Translation Look-Aside Buffer Low Register

TLBHI: Translation Look-Aside Buffer High Register

TLBX: Translation Look-Aside Buffer Index Register

TLBSX: Translation Look-Aside Buffer Search Index Register

All these registers are accessed via MFS and MTS instructions.

Access TLB Entries

First of all, we need to know the TLB entry structure. A TLB entry can be accessed via TLBLO and TLBHI registers, with an index set at TLBX.


RPN (Real Page Number or Physical Page Number) is used to form physical address. Depending on the value of page size, certain bits of RPN are not used in physical address.

TAG (TLB-entry tag) is compared with the virtual memory address under control of SIZE.

SIZE (page Size) specifies page size.

There are 64 entries in the unified TLB in microblaze MMU architecture. So the INDEX field (bit 26 to 31, as shown below) of TLBX register is used as index of MMU entries. The field is also updated when TLBSX register is written with a virtual address and the virtual address is found in TLB entries.


Software can read and write unified TLB entries by using the TLBX register index (numbered 0 to 63) corresponding to one of the 64 entries in the TLB. The tag and data portions are read and written separately, so software must execute two MFS or MTS instructions to completely access an entry. The TLB is searched for a specific translation using the TLBSX register. TLBSX locates a translation using an effective address and loads the corresponding UTLB index into the TLBX register.


All the above are the story about MMU architecture of microblaze. You may refer to xilinx document: MicroBlaze Processor Reference Guide for more detail explanation on microblaze and its instructions. The figures and information in this blog comes from the document directly. Linux kernel set up memory page mapping in arch/microblaze/head.S. I am going to talk about the codes in head.S in next blog in this series.

WordPress Tags: physical address,virtual address,TLB,Xilinx,FPGA,microblaze,MMU,Linux,Memory Management


what you need to Run Linux on XILINX FPGA

To run a kernel on a piece of hardware, you need several tools and a stable environment to do your job in.
Here is my tool lists, with a hyperlink so that you may easy to download them and repeat my job



Let’s just begin with environment.
I recommend doing your entire job in Linux OS. To compile a linux kernel in a linux environment is the safest way to do, and fastest.
There are plenty of distributions for Linux out there for you to choose.
According to my experience with different distributions, here is my recommendation:

1. RHEL/CentOS 5.x or 6.x

Although in different names, RHEL (RedHat Enterprise Linux) and CentOS are basically the same. They share same rpm repositories, same kernel version, just named differently.
It is the environment that is most stable and hardly ever goes wrong. It may be a little bit slow to run on your laptop, but it’s the first choice for a beginner.

2. Ubuntu

I know a lot of people who are fond of Ubuntu, because of its glorious desktop. As more and more people are switching to it, it has a great support community on-line. You can easily find any solution to your problem. If you want a linux system on your personal laptop, it will be a good choice.

3. Debian

For people who have experience with Linux, I would like to recommend Debian. I am personally fond of Debian, because of its speed in compilation. Debian is the fastest system for software compilation. It takes one thirds time to compile a kernel than Fedora. However, as not many people work on it, you may need to solve your problems alone. So, if you are a linux guru, pick this one
Another important thing to mention is that you’d better have enough space to build your kernel source. Xilinx Tools may cost you about 10GB of your file system and the kernel source folder after compilation will cost 1GB. When you have more than one kernel sources in your system… that’s quite a lot space…


Tool Chain

There is a saying that if you want to do a job right, you need to find proper tools. Here is what you need to build kernel on your FPGA board.

1. Xilinx Design Suite

That’s definitely what you need to build up an microblaze system on your FPGA and that’s the piece of hardware you will run you kernel on.
The version of Xilinx DS I recommended currently are 12.4 or 13.2.
It can be downloaded from http://www.xilinx.com
Well, I have to say, the installation package gets bigger and bigger these days… You’d better have enough space for that.

2. Cross-compilation Tool Chain

As you are going to compile linux kernel for Xilinx Microblaze instead Intel CPU, you need a cross-platform compilation tool chain to do the work for you. The reason that they are named “cross-compilation” is that they themselves run on x86 or x64 systems while their output binaries run on other platforms.
For cross-compilation tool chain we use here, there are two places you may get them:

a> ${XILINX_DS_DIR}/EDK/gnu/microblaze/lin(64)/bin/mb-*
These gnu compilation collection work for microblaze (Big-Endian) by default. If your design is an AXI based system, you need to add –mlittle-endian to compilation flags and –EL to link flags.

b> They are available at http://git.xilinx.com
You may download and compile them for your own system.

3. Kernel Resource

Actually, xilinx open source wiki keeps a neat and clean tree for linux kernel. the source files can be download at: http://git.xilinx.com
For details, you may refer to xilinx open source wiki at http://xilinx.wikidot.com

4. Device Tree

To make your kernel know about your hardware, especially how the virtual memory of CPU are organized and which part of physical memory each peripheral device locates at, you need device tree to get all these information for kernel. The device tree for Xilinx EDK is available at: http://git.xilinx.com . It will generate a dts file for your hardware automatically and checks out the necessary peripheral devices necessary for linux kernel.


Requirement on Hardware

For the hard ware system you want to run linux on, here are a set of things you have to instantiate in your design:

1> a memory over 10 MBytes.

You need space to hold kernel image and stacks for processes. However, I am still checking on the exactly minimum size of memory needed.

2> Interrupt controller

You definitely need this. This is how OS interface with outside world and avoids busy waiting.

3> Timer

Without timer, OS cannot do scheduling.

4> Uart

This is the easiest way to interact with your kernel, and see what’s going on.

5> you may add other ip cores you want or your custom ones.

Besides all these peripherals, you also need to configure microblaze.

1> Enable MMU

In early days, embedded linux system always claim that a kernel without MMU is its advantage, as it runs fast and works tight with hardware registers underneath. However, as embedded systems run faster, embedded OS’s are able to provide better memory management functionalities to user processes. As stated clearly on xilinx open source wiki, MMU is required with 2 protection zones available.

2> cache

cache is needed for synchronous SRAM or SDRAM to make the system run faster.

3> FPU, MULT, SHIFT, etc.

add and configure as you like. Certainly, they can be helpful.

After settle down all these, you can begin to build your kernel for your hardware now.

WordPress Tags: Cross-Compilation,device tree,Linux,XILINX,FPGA,kernel,Microblaze,Hardware

How-To Debug Linux Kernel with Xilinx FPGA

To compile and successfully boot an operating on a piece of hardware is an interesting job. Anyone who is interested in understanding how a computer system work will always find it fascinating to run a full OS kernel on a hardware that you create yourself.

Back to the days when I studied linux kernel in my senior years, the first thing I got to do is to compile a linux kernel on your own laptop. It is very likely that the kernel panics during the boot sequence. Well, as it is the kernel, you have no way to debug but reboot your computer, browse through the internet to see which configuration you missed or misconfigured, reconfigure the kernel, and wait another hour to get it recompiled.

These days, I am trying to run linux kernel on Digilent Atlys FPGA boards (with Xilinx Spartan6 LX45 FPGA on it). With the help of Xilinx Design Suites and the kernel maintained at http://git.xilinx.com. It will be easy to do debug for Linux kernel and understand how the kernel works.

Here is the procedure to follow to debug linux kernel on Xilinx FPGAs:

Ø Start two Xilinx Bash shell. One as a debug port that downloading linux image onto Atlys, while the other as the gdb debugging interface.

Ø Browse to where your linux kernel is located. run xmd.
Connect to atlys board by typing:

% connect mb mdm -cable type xilinx_plugin modulename digilent_plugin

Note: A GDB server will open on localhost:1234 with tcp protocol.


Ø Dowload the kernel image onto FPGA

% dow simpleImage.Atlys_mmu


Ø In the other Xilinx bash shell, go to the root directory of the kernel you built.
Launch mb-gdb by

% mb-gdb -nw vmlinux


Ø Connect to localhost:1234 for remote debug.

(gdb) target remote localhost:1234


Now we have set up the debugging tool chane for Linux Debugging on FPGA.


WordPress Tags: MMU,Debug,Linux,Kernel,Xilinx,FPGA,Digilent,Atlys,Microblaze