Thursday, August 26, 2010

Jiffies: INTC and GPT timer crash

Ok, So after configuring INTC and GPT1, its now possible to proceed a bit further, where in I wait looping inside calibration_loop function waiting for jiffies to be updated.

Sure enough, I do receive the GPT1 interrupt when the timer overflows, however, I see a big crash happening as the interrupt is not getting correctly mapped.

Also, it must be noted that GPT1 is still not configured correctly for 1 ms delay, but rather need to wait for more than 10 min to get this crash.

Below is a log that I get when system crashes:


OMAP3 beagleboard.org # bootm 0x80300000
## Booting kernel from Legacy Image at 80300000 ...
   Image Name:   Puppy Linux
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    1399776 Bytes =  1.3 MB
   Load Address: 80008000
   Entry Point:  80008000
   Verifying Checksum ... OK
   Loading Kernel Image ... OK
OK


Starting kernel ...


Uncompressing Linux... done, booting the kernel.
<5>Linux version 2.6.35-rc3 (amar@amar-laptop) (gcc version 4.4.1 (Sourcery G++                                                                                                              Lite 2010q1-188) ) #12 PREEMPT Thu Aug 26 23:04:06 CEST 2010
CPU: ARMv7 Processor [411fc083] revision 3 (ARMv7), cr=10c53c7f
CPU: VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
Machine: PuppyBeagle
Memory policy: ECC disabled, Data cache writeback
<7>On node 0 totalpages: 65536
<7>free_area_init_node: node 0, pgdat c02d4714, node_mem_map c02eb000
<7>  Normal zone: 512 pages used for memmap
<7>  Normal zone: 0 pages reserved
<7>  Normal zone: 65024 pages, LIFO batch:15
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 65024
<5>Kernel command line: console=ttyS2,115200n8 console=tty0 root=/dev/mmcblk0p2                                                                                                              rw rootfstype=ext3 rootwait omapfb.video_mode=1024x768MR-16@60
<6>PID hash table entries: 1024 (order: 0, 4096 bytes)
<6>Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
<6>Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
<6>Memory: 128MB 128MB = 256MB total
<5>Memory: 256916k/256916k available, 5228k reserved, 0K highmem
<5>Virtual kernel memory layout:
    vector  : 0xffff0000 - 0xffff1000   (   4 kB)
    fixmap  : 0xfff00000 - 0xfffe0000   ( 896 kB)
    DMA     : 0xffc00000 - 0xffe00000   (   2 MB)
    vmalloc : 0xd0800000 - 0xe0000000   ( 248 MB)
    lowmem  : 0xc0000000 - 0xd0000000   ( 256 MB)
    modules : 0xbf000000 - 0xc0000000   (  16 MB)
      .init : 0xc0008000 - 0xc0021000   ( 100 kB)
      .text : 0xc0021000 - 0xc02a9000   (2592 kB)
      .data : 0xc02c0000 - 0xc02d4d20   (  84 kB)
<6>Hierarchical RCU implementation.
<6>     RCU-based detection of stalled CPUs is disabled.
<6>     Verbose stalled-CPUs detection is disabled.
<6>NR_IRQS:96

Console: colour dummy device 80x30
<6>console [tty0] enabled
<6>Calibrating delay loop... <4>Bad IRQ3224117184
<1>Unhandled fault: alignment exception (0x801) at 0xfc200087
<0>Internal error: : 801 [#1] PREEMPT
<0>last sysfs file:
Modules linked in:
CPU: 0    Not tainted  (2.6.35-rc3 #12)
PC is at puppy_irq_mask+0x2c/0x34
LR is at handle_level_irq+0x3c/0x170
pc : []    lr : []    psr: 600001d3
sp : c02c1f58  ip : cfc07220  fp : 00000000
r10: 0000001f  r9 : c02c0000  r8 : 00000002
r7 : 00000003  r6 : 00000003  r5 : 00000000  r4 : c02c64d4
r3 : fc200087  r2 : 00000001  r1 : c02c64d4  r0 : 00000008
Flags: nZCv  IRQs off  FIQs off  Mode SVC_32  ISA ARM  Segment kernel
Control: 10c5387f  Table: 80004019  DAC: 00000017
<0>Process swapper (pid: 0, stack limit = 0xc02c0268)
<0>Stack: (0xc02c1f58 to 0xc02c2000)
<0>1f40:                                                       00000003 00000000
<0>1f60: c001bce8 c0021078 60000153 ffffffff c02c1fac c0021ae4 00002000 c02c1f30
<0>1f80: ffff8ad0 ffff8ad0 c02c5718 c02c3938 c001bce8 c02c3ad0 8001aaa4 411fc083
<0>1fa0: 0000001f 00000000 cfc07254 c02c1fc0 c0019c08 c0019c14 60000153 ffffffff
<0>1fc0: c02d4da0 c001b44c c001bce8 c02c3ad0 8001aaa4 c0008b04 c0008660 00000000
<0>1fe0: 00000000 c001bce8 10c53c7d c02d50b0 c001bce4 80008034 00000000 00000000
[] (puppy_irq_mask+0x2c/0x34) from [] (handle_level_irq+0x3c                                                                                                             /0x170)
[] (handle_level_irq+0x3c/0x170) from [] (asm_do_IRQ+0x78/0x                                                                                                             8c)
[] (asm_do_IRQ+0x78/0x8c) from [] (__irq_svc+0x44/0x78)
Exception stack(0xc02c1f78 to 0xc02c1fc0)
1f60:                                                       00002000 c02c1f30
1f80: ffff8ad0 ffff8ad0 c02c5718 c02c3938 c001bce8 c02c3ad0 8001aaa4 411fc083
1fa0: 0000001f 00000000 cfc07254 c02c1fc0 c0019c08 c0019c14 60000153 ffffffff
[] (__irq_svc+0x44/0x78) from [] (calibrate_delay+0xbc/0x1e4                                                                                                             )
[] (calibrate_delay+0xbc/0x1e4) from [] (start_kernel+0x228/                                                                                                             0x27c)
[] (start_kernel+0x228/0x27c) from [<80008034>] (0x80008034)
<0>Code: e203340f e0823003 e3a02001 e1a00012 (e5830000)
<4>Bad IRQ3224116728
<4>Bad IRQ65542
<4>Bad IRQ65542
<4>Bad IRQ65542
<4>Bad IRQ65542
<4>Bad IRQ65542
<4>Bad IRQ65542
<4>Bad IRQ65542
<4>Bad IRQ65542




Next steps, configure GPT for 1 ms.
Mapping GPT interrupt with proper interrupt handler. A lot of interrupt handling study to be done here!

Wednesday, August 25, 2010

Printascii fix

Same function also points to yet another function
mdesc->map_io()
which actually does mapping of the statically mapped devices.
/*
 * Ask the machine support to map in the statically mapped devices.
 */

This ended at puppy_map_io()

BINGO!!

Probably this is where I need to add descriptor for my UART3

Currently mapping only this:

#define PUPPY_L4_BASE_VIRT 0xFC000000
#define L4_CORE_SIZE 0x00FFFFFF




static struct map_desc puppy_io_desc[] __initdata = {


{
                .virtual        = PUPPY_L4_BASE_VIRT,
                .pfn            = __phys_to_pfn(PUPPY_L4_BASE),
                .length         = L4_CORE_SIZE,
                .type           = MT_DEVICE
        },
};


Add entry for our UART3, which exists under L4_PER_BASE region. New mach descriptor now looks like this:


static struct map_desc puppy_io_desc[] __initdata = {


{
                .virtual        = PUPPY_L4_CORE_VIRT,
                .pfn            = __phys_to_pfn(PUPPY_L4_BASE),
                .length         = L4_CORE_SIZE,
                .type           = MT_DEVICE
        },
{
                .virtual        = PUPPY_L4_PER_VIRT,
                .pfn            = __phys_to_pfn(PUPPY_L4_PER_BASE),
                .length         = L4_PER_SIZE,
                .type           = MT_DEVICE
        },
};

where
#define PUPPY_L4_PER_VIRT 0xFD000000
#define L4_PER_SIZE 0x000FFFFF

and

#define PUPPY_L4_PER_BASE 0x49000000



After rebuilding the image, I did manage to get all the logs without anymore crashes :)

However, I still need to fix the calibration loop issues occuring due to non jiffy timer implementation. The new log now looks like this:


## Booting kernel from Legacy Image at 80300000 ...
   Image Name:   Puppy Linux
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    1398532 Bytes =  1.3 MB
   Load Address: 80008000
   Entry Point:  80008000
   Verifying Checksum ... OK
   Loading Kernel Image ... OK
OK


Starting kernel ...


Uncompressing Linux... done, booting the kernel.
<5>Linux version 2.6.35-rc3 (amar@amar-laptop) (gcc version 4.4.1 (Sourcery G++              Lite 2010q1-188) ) #7 PREEMPT Wed Aug 25 23:04:33 CEST 2010
CPU: ARMv7 Processor [411fc083] revision 3 (ARMv7), cr=10c53c7f
CPU: VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
Machine: PuppyBeagle
Memory policy: ECC disabled, Data cache writeback
<7>On node 0 totalpages: 65536
<7>free_area_init_node: node 0, pgdat c02d4714, node_mem_map c02eb000
<7>  Normal zone: 512 pages used for memmap
<7>  Normal zone: 0 pages reserved
<7>  Normal zone: 65024 pages, LIFO batch:15
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 65024
<5>Kernel command line: console=ttyS2,115200n8 console=tty0 root=/dev/mmcblk0p2              rw rootfstype=ext3 rootwait omapfb.video_mode=1024x768MR-16@60
<6>PID hash table entries: 1024 (order: 0, 4096 bytes)
<6>Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
<6>Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
<6>Memory: 128MB 128MB = 256MB total
<5>Memory: 256916k/256916k available, 5228k reserved, 0K highmem
<5>Virtual kernel memory layout:
    vector  : 0xffff0000 - 0xffff1000   (   4 kB)
    fixmap  : 0xfff00000 - 0xfffe0000   ( 896 kB)
    DMA     : 0xffc00000 - 0xffe00000   (   2 MB)
    vmalloc : 0xd0800000 - 0xe0000000   ( 248 MB)
    lowmem  : 0xc0000000 - 0xd0000000   ( 256 MB)
    modules : 0xbf000000 - 0xc0000000   (  16 MB)
      .init : 0xc0008000 - 0xc0021000   ( 100 kB)
      .text : 0xc0021000 - 0xc02a9000   (2592 kB)
      .data : 0xc02c0000 - 0xc02d4d20   (  84 kB)
<6>Hierarchical RCU implementation.
<6>     RCU-based detection of stalled CPUs is disabled.
<6>     Verbose stalled-CPUs detection is disabled.
<6>NR_IRQS:96
Console: colour dummy device 80x30
<6>console [tty0] enabled
<6>Calibrating delay loop...

Note that UART3 is still mapped at the virtual address : 0xFD000000

Next steps:

  • Understand the mem. layout printed in the above log.
  • Fixing jiffies

Printascii failure

I added functional support for printascii after editing vprintk(), which will be described in a seperate posting. But doing so resulted in system printing debug messages until


OMAP3 beagleboard.org # bootm 0x80300000
## Booting kernel from Legacy Image at 80300000 ...
   Image Name:   Puppy Linux
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    1398532 Bytes =  1.3 MB
   Load Address: 80008000
   Entry Point:  80008000
   Verifying Checksum ... OK
   Loading Kernel Image ... OK
OK

Starting kernel ...

Uncompressing Linux... done, booting the kernel.
<5>Linux version 2.6.35-rc3 (amar@amar-laptop) (gcc version 4.4.1 (Sourcery G++ Lite 2010q1-188) ) #6 PREEMPT Wed Aug 25 00:38:20 CEST 2010
CPU: ARMv7 Processor [411fc083] revision 3 (ARMv7), cr=10c53c7f
CPU: VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
Machine: PuppyBeagle
Memory policy: ECC disabled, Data cache writeback
<7>On node 0 totalpages: 65536
<7>free_area_init_node: node 0, pgdat c02d4714, node_mem_map c02eb000
<7>  Normal zone: 512 pages used for memmap
<7>  Normal zone: 0 pages reserved
<7>  Normal zone: 65024 pages, LIFO batch:15

But after this, the system started crashing. Careful debugging revealed that system crashed inside
setup_arch( ) call. Further debugging revealed the following:

clean_pmd_entry clears 0xfd000000 (VA for UART3)

and then returns back from __phys_to_pfn call within clean_pmd_entry fn. (inline fn..no symbol found)
called from devicemaps_init() inside mmu.c

devicemaps_init( ) banner clearly states that:
"We clear out pg tables for all mappings above VMALLOC_END, we will remove any debug device mappings.
This means that you have to be careful how you debug this function, or any called function. This means that
you can't use any function or debugging method which may touch the device, otherwise the kernel *_will_ crash *!!!

Exact behavior as seen in my case!

Now I know......

Next step: Probably remove debug printascii support and find any other way of retrieving the prints.....

Tuesday, August 3, 2010

Static IO Remapping during kernel bootup

Even though it is possible to map all IO modules, it becomes redundant to map all modules to VM.
See the mail discussion from RMK regarding this!

So, we do boot time IO mapping only for the modules that becomes absolutely necessary.

In our case, this happens to be the INTC and the GPT1.

In order to do this mapping, we first define the following structure in our core.c file:

static struct map_desc puppy_io_desc[] __initdata = {

    {
                .virtual        = PUPPY_L4_BASE_VIRT,
                .pfn            = __phys_to_pfn(PUPPY_L4_BASE),
                .length         = L4_CORE_SIZE,
                .type           = MT_DEVICE
        },
};



Note that we can define any VA for our peripherals, as long as they exist above VMALLOC_END, which again, is defined by us in vmalloc.h file
Check this discussion.

We chose:

#define PUPPY_L4_BASE_VIRT    0xFC000000
#define L4_CORE_SIZE                  0x00FFFFFF

While IO_ADDRESS in hardware.h file is defined as:

#define IO_ADDRESS(x) (((x) & 0x0fffffff) + (((x) >> 4) & 0x0f000000) + 0xf0000000)

With this configuration, we are now able to program INTC, though GPT1 is still not getting configured correctly. I see that GPT1 is by default free-running even before it is programmed and programming it seems to be ignored. May also need to cross check INTC programming as well.

All in all, it is now possible to progress further as I can ensure correct register locations are being accessed with the above settings....phew, finally!

[Notes] Final Page Table Directory

Created @0xC0007000 for the first bank (0xC0000000 - 0xC8000000 :: 0x80000000 - 0x88000000)
Second created @0xC0007200 upto @0xC00073FF for the first bank (0xC8000000 - 0xD0000000 :: 0x88000000 - 0x90000000)


Kernel starts @0xc0008000 while PTE offset is available @0xC0007FF8

PTE for IO MAP Done @0xC0007000
Since Virtual addr was designed to start @0xFC000000 and size is defined in MACH DESCRIPTOR as
0x7FFFFFFF, we seem to overlap and end @0x04000000!