KITL only working in Polling Mode (CE 6.0 on an x86 CEPC)

Sorry, this post is long.

I've successfully built a Windows CE 6.0 OS for my x86 "CEPC", and can
download it to my target successfully over ethernet. I can run my simple
"Hello World" applications on the target with no problem.

However, I have been able get the KITL debugger to work in "Polling Mode"
only. (To do that, I modified the kitl_x86.c file to add the
OAL_KITL_FLAGS_POLL to the pKitlArgs->flags setting in the
InitKitlEtherArgs() function.) If I try to use the default "Interrupt Mode"
(that is, without setting the _POLL flag), then my OS image will hang during
the booting stage, with a repeating sequence (every 8 seconds) of 16
ethernet UDP packets being sent from the target to the host, which never
seem to get acknowledged by PB. Each UDP packet's data payload starts with
"EDBG".

Here are the last several messages that are received over the serial
connection to the target:
= = = = = = = = =
<<<EARLIER MESSAGES SNIPPED>>>
EDBG:NE2000Init using I/O range at 0x00000300
EDBG:NE2000:HWRamTest: srambase: 0x00004000, sramsize: 0x00004000, pstart:
0x46
Ne2kDbg:: HWSetMCRegs():: Set all to 0x00
-EDBG:NE2000Init
x86KitlCreateName: Using Device Name 'CEPCLS53401'
OALKitlCreateName: Using Device Name 'CEPCLS53401'
KITL: *** Device Name CEPCLS53401 ***
KITL: using sysintr 0x1A
KITL: DHCP get/renew device IP: 1
VBridge:: built on [Sep 6 2006] time [19:21:34]
VBridgeInit()...TX = [16384] bytes -- Rx = [16384] bytes
Tx buffer [0xA0C18500] to [0xA0C1C500].
Rx buffer [0xA0C1C520] to [0xA0C20520].
VBridge:: NK add MAC: [0-40-5-6C-D0-99]
Connecting to Desktop
KITL: Connected host IP: 1 Port: 1419
KeyIndex 0 = -1
KeyIndex 1 = -1
KeyIndex 2 = -1
KeyIndex 3 = -1
KeyIndex 4 = -1
KeyIndex 5 = -4194305
KeyIndex 6 = -4194305
KeyIndex 7 = -1
+KITLRegisterDfltClient, service:0
+KITLRegisterDfltClient, service:1
Waiting for service 'DBGMSG' to connect..., fUseSysCalls = 0
ProcessAdminMsg: Receive Config message for service DBGMSG
ProcessAdminMsg: Receive Config message for service PPSH
ProcessAdminMsg: Receive Config message for service DBGMSG
!Received config for unrecognized service KDBG
ProcessAdminMsg: Receive Config message for service DBGMSG
ProcessAdminMsg: Receive Config message for service PPSH
!Received config for unrecognized service KDBG
!Received config for unrecognized service KDBG
+KITLRegisterDfltClient, service:2
ProcessAdminMsg: Receive Config message for service KDBG
ProcessAdminMsg: Receive Config message for service KDBG
KdpDeleteBreakpoint: Invalid handle
KdpDeleteBreakpoint: Invalid handle
<<<THE ABOVE MESSAGE REPEATS ABOUT 160 TIMES>>>
KITL: Leaving polling mode... 0x80C08060
KITL Checking client registrations
+RegisterClientPart2: Id 0x00000000
KITL Timer thread started, (hTh: 0x00520002, pTh: 0x83FE3C18)
Closing Handle of Timer Thread
-RegisterClientPart2 returns 1
+RegisterClientPart2: Id 0x00000001
-RegisterClientPart2 returns 1
KITL Creating IST
KITL Interrupt thread started (hTh: 0x005A0002, pTh: 0x83FC1414), using
SYSINTR 26
Enabling adapter ints...
Reserve KITL IRQ: No IRQ reserved, KITL NIC IRQ may be shared with other
devices.
0x680002 (0x102): KITLSend(0): Timed out waiting for ack (AckExpected: 74,
TxSeq: 90)
0x930002 (0x102): KITLSend(0): Timed out waiting for ack (AckExpected: 74,
TxSeq: 90)
0x680002 (0x102): KITLSend(0): Timed out waiting for ack (AckExpected: 74,
TxSeq: 90)
<<<THE ABOVE MESSAGE REPEATS FOREVER, ABOUT EVERY 10 SECONDS>>>
= = = = = = = =

And here are the last several lines of Debug Output text from the Visual
Studio window:

= = = = = = = =
<<<EARLIER MESSAGES SNIPPED>>>
4294767914 PID:400002 TID:680002 OSAXST1: >>> Loading Module 'romfsd.dll'
(0x83FA024C) at address 0xC01B0000-0xC01B6000 in Process 'NK.EXE'
(0x80C23C80)
4294767928 PID:400002 TID:680002 FSDMGR!MountTable_t::RegisterVolumeName:
Registered "ROM" at index 3
4294767929 PID:400002 TID:680002 FSDMGR!MountTable_t::RegisterVolume:
Registered volume at index 3 (Name="ROM", MountFlags=0x71)
4294767992 PID:400002 TID:680002 Succeeded on CheckFile
4294767993 PID:400002 TID:680002 RestoreFile(0x008f0003) called fDelete =
TRUE
4294768145 PID:400002 TID:680002 FSDMGR!STOREMGR_StartBootPhase BootPhase=0
(PrevBootPhase=-1)
4294768146 PID:400002 TID:680002 FSDMGR: File security disabled.
4294768147 PID:400002 TID:680002 FSDMGR!AutoLoadFileSystems:
CurrentBootPhase=0, LoadFlags=1
4294768148 PID:400002 TID:680002 FSDMGR!AutoLoadFileSystem:
CurrentBootPhase=0, RootKey=System\StorageManager\AutoLoad,
FileSystem_t=ObjectStore
4294768149 PID:400002 TID:680002 FILESYS: RAM File System FSD_MountDisk
registering folder "Object Store"
4294768150 PID:400002 TID:680002 FSDMGR!MountTable_t::RegisterVolumeName:
Registered "Object Store" at index 4
4294768150 PID:400002 TID:680002 FSDMGR!MountTable_t::RegisterVolume:
Registered volume at index 4 (Name="", MountFlags=0x46)
4294768152 PID:400002 TID:680002 PCIReg(key:
Drivers\BuiltIn\PCI\Instance\KITL, device: 0/3/0)
4294768175 PID:400002 TID:680002 Override Serial Driver: serial debug output
selected on COM port 1, disabling serial driver for that port.
4294768175 PID:400002 TID:680002 Override Serial Driver: No COM port
selected for serial KITL transport, no override necessary.
4294768176 PID:400002 TID:680002 FSDMGR!STOREMGR_StartBootPhase BootPhase=1
(PrevBootPhase=0)
4294768176 PID:400002 TID:680002 FSDMGR!AutoLoadFileSystems:
CurrentBootPhase=1, LoadFlags=1
<<<THE ABOVE MESSAGE IS THE LAST MESSAGE THAT APPEARS>>>
= = = = = = = =

I have confirmed that the Target>Connectivity Options>Kernel Service
Map>Debugger is set to KdStub, and the Download and Transport settings are
set to Ethernet.

Perhaps my target system has some KITL interrupt/IST issue, perhaps related
to SYSINTR 26? Or perhaps PB is hanging for some different reason, and it
stops acknowledging the packets sent by the target? I had read in other
postings that it could be an MTU size issue on my development workstation; I
haven't investigated this issue yet. But it's interesting that if I switch
to Polling Mode, everything starts up fine and I can execute my applications
on my CE target.

Has anyone seen this problem and then been able to get KITL working in
Interrupt Mode?

I can live with Polling Mode for now (in fact, I don't even know if
Interrupt Mode will improve the performance of my system, since I've never
seen it work before), but I wondered if anyone has found a fix for this
problem.

As always, any guidance is greatly appreciated!

David K

Vinz
Fri Nov 02 04:49:01 PDT 2007

Hi David,

I have the exact same behavior on my x86 BSP on ce 6.0. I know that the code
I use for my KITL is ok since it worked in CE 5.0. I work on this interrupt
problem for 2 weeks!!! I haven't found a clue.

If you find a solution please share it with me (I'll do the same).

"David K in San Jose" wrote:

> Sorry, this post is long.
>
> I've successfully built a Windows CE 6.0 OS for my x86 "CEPC", and can
> download it to my target successfully over ethernet. I can run my simple
> "Hello World" applications on the target with no problem.
>
> However, I have been able get the KITL debugger to work in "Polling Mode"
> only. (To do that, I modified the kitl_x86.c file to add the
> OAL_KITL_FLAGS_POLL to the pKitlArgs->flags setting in the
> InitKitlEtherArgs() function.) If I try to use the default "Interrupt Mode"
> (that is, without setting the _POLL flag), then my OS image will hang during
> the booting stage, with a repeating sequence (every 8 seconds) of 16
> ethernet UDP packets being sent from the target to the host, which never
> seem to get acknowledged by PB. Each UDP packet's data payload starts with
> "EDBG".
>
> Here are the last several messages that are received over the serial
> connection to the target:
> = = = = = = = = =
> <<<EARLIER MESSAGES SNIPPED>>>
> EDBG:NE2000Init using I/O range at 0x00000300
> EDBG:NE2000:HWRamTest: srambase: 0x00004000, sramsize: 0x00004000, pstart:
> 0x46
> Ne2kDbg:: HWSetMCRegs():: Set all to 0x00
> -EDBG:NE2000Init
> x86KitlCreateName: Using Device Name 'CEPCLS53401'
> OALKitlCreateName: Using Device Name 'CEPCLS53401'
> KITL: *** Device Name CEPCLS53401 ***
> KITL: using sysintr 0x1A
> KITL: DHCP get/renew device IP: 1
> VBridge:: built on [Sep 6 2006] time [19:21:34]
> VBridgeInit()...TX = [16384] bytes -- Rx = [16384] bytes
> Tx buffer [0xA0C18500] to [0xA0C1C500].
> Rx buffer [0xA0C1C520] to [0xA0C20520].
> VBridge:: NK add MAC: [0-40-5-6C-D0-99]
> Connecting to Desktop
> KITL: Connected host IP: 1 Port: 1419
> KeyIndex 0 = -1
> KeyIndex 1 = -1
> KeyIndex 2 = -1
> KeyIndex 3 = -1
> KeyIndex 4 = -1
> KeyIndex 5 = -4194305
> KeyIndex 6 = -4194305
> KeyIndex 7 = -1
> +KITLRegisterDfltClient, service:0
> +KITLRegisterDfltClient, service:1
> Waiting for service 'DBGMSG' to connect..., fUseSysCalls = 0
> ProcessAdminMsg: Receive Config message for service DBGMSG
> ProcessAdminMsg: Receive Config message for service PPSH
> ProcessAdminMsg: Receive Config message for service DBGMSG
> !Received config for unrecognized service KDBG
> ProcessAdminMsg: Receive Config message for service DBGMSG
> ProcessAdminMsg: Receive Config message for service PPSH
> !Received config for unrecognized service KDBG
> !Received config for unrecognized service KDBG
> +KITLRegisterDfltClient, service:2
> ProcessAdminMsg: Receive Config message for service KDBG
> ProcessAdminMsg: Receive Config message for service KDBG
> KdpDeleteBreakpoint: Invalid handle
> KdpDeleteBreakpoint: Invalid handle
> <<<THE ABOVE MESSAGE REPEATS ABOUT 160 TIMES>>>
> KITL: Leaving polling mode... 0x80C08060
> KITL Checking client registrations
> +RegisterClientPart2: Id 0x00000000
> KITL Timer thread started, (hTh: 0x00520002, pTh: 0x83FE3C18)
> Closing Handle of Timer Thread
> -RegisterClientPart2 returns 1
> +RegisterClientPart2: Id 0x00000001
> -RegisterClientPart2 returns 1
> KITL Creating IST
> KITL Interrupt thread started (hTh: 0x005A0002, pTh: 0x83FC1414), using
> SYSINTR 26
> Enabling adapter ints...
> Reserve KITL IRQ: No IRQ reserved, KITL NIC IRQ may be shared with other
> devices.
> 0x680002 (0x102): KITLSend(0): Timed out waiting for ack (AckExpected: 74,
> TxSeq: 90)
> 0x930002 (0x102): KITLSend(0): Timed out waiting for ack (AckExpected: 74,
> TxSeq: 90)
> 0x680002 (0x102): KITLSend(0): Timed out waiting for ack (AckExpected: 74,
> TxSeq: 90)
> <<<THE ABOVE MESSAGE REPEATS FOREVER, ABOUT EVERY 10 SECONDS>>>
> = = = = = = = =
>
> And here are the last several lines of Debug Output text from the Visual
> Studio window:
>
> = = = = = = = =
> <<<EARLIER MESSAGES SNIPPED>>>
> 4294767914 PID:400002 TID:680002 OSAXST1: >>> Loading Module 'romfsd.dll'
> (0x83FA024C) at address 0xC01B0000-0xC01B6000 in Process 'NK.EXE'
> (0x80C23C80)
> 4294767928 PID:400002 TID:680002 FSDMGR!MountTable_t::RegisterVolumeName:
> Registered "ROM" at index 3
> 4294767929 PID:400002 TID:680002 FSDMGR!MountTable_t::RegisterVolume:
> Registered volume at index 3 (Name="ROM", MountFlags=0x71)
> 4294767992 PID:400002 TID:680002 Succeeded on CheckFile
> 4294767993 PID:400002 TID:680002 RestoreFile(0x008f0003) called fDelete =
> TRUE
> 4294768145 PID:400002 TID:680002 FSDMGR!STOREMGR_StartBootPhase BootPhase=0
> (PrevBootPhase=-1)
> 4294768146 PID:400002 TID:680002 FSDMGR: File security disabled.
> 4294768147 PID:400002 TID:680002 FSDMGR!AutoLoadFileSystems:
> CurrentBootPhase=0, LoadFlags=1
> 4294768148 PID:400002 TID:680002 FSDMGR!AutoLoadFileSystem:
> CurrentBootPhase=0, RootKey=System\StorageManager\AutoLoad,
> FileSystem_t=ObjectStore
> 4294768149 PID:400002 TID:680002 FILESYS: RAM File System FSD_MountDisk
> registering folder "Object Store"
> 4294768150 PID:400002 TID:680002 FSDMGR!MountTable_t::RegisterVolumeName:
> Registered "Object Store" at index 4
> 4294768150 PID:400002 TID:680002 FSDMGR!MountTable_t::RegisterVolume:
> Registered volume at index 4 (Name="", MountFlags=0x46)
> 4294768152 PID:400002 TID:680002 PCIReg(key:
> Drivers\BuiltIn\PCI\Instance\KITL, device: 0/3/0)
> 4294768175 PID:400002 TID:680002 Override Serial Driver: serial debug output
> selected on COM port 1, disabling serial driver for that port.
> 4294768175 PID:400002 TID:680002 Override Serial Driver: No COM port
> selected for serial KITL transport, no override necessary.
> 4294768176 PID:400002 TID:680002 FSDMGR!STOREMGR_StartBootPhase BootPhase=1
> (PrevBootPhase=0)
> 4294768176 PID:400002 TID:680002 FSDMGR!AutoLoadFileSystems:
> CurrentBootPhase=1, LoadFlags=1
> <<<THE ABOVE MESSAGE IS THE LAST MESSAGE THAT APPEARS>>>
> = = = = = = = =
>
> I have confirmed that the Target>Connectivity Options>Kernel Service
> Map>Debugger is set to KdStub, and the Download and Transport settings are
> set to Ethernet.
>
> Perhaps my target system has some KITL interrupt/IST issue, perhaps related
> to SYSINTR 26? Or perhaps PB is hanging for some different reason, and it
> stops acknowledging the packets sent by the target? I had read in other
> postings that it could be an MTU size issue on my development workstation; I
> haven't investigated this issue yet. But it's interesting that if I switch
> to Polling Mode, everything starts up fine and I can execute my applications
> on my CE target.
>
> Has anyone seen this problem and then been able to get KITL working in
> Interrupt Mode?
>
> I can live with Polling Mode for now (in fact, I don't even know if
> Interrupt Mode will improve the performance of my system, since I've never
> seen it work before), but I wondered if anyone has found a fix for this
> problem.
>
> As always, any guidance is greatly appreciated!
>
> David K
>
>

Jimmy
Thu Nov 08 00:45:00 PST 2007

Mines too..
KITL: Connected host IP: 1 Port: 1360
KITL: Leaving polling mode... 0x80BAC080
Closing Handle of Timer Thread
+RTL8139EnableInts()..
-RTL8139EnableInts()..
Reserve KITL IRQ: No IRQ reserved, KITL NIC IRQ may be shared with other
devices
.

It seems kitl does not work on interrupt mode.

David
Tue Nov 13 00:58:26 PST 2007

Following up, in case anyone has been following this thread...

I was finally able to get KITL to run in Interrupt Mode, by switching to a
different ethernet card. Instead of using the (ancient) Linksys LNE2000 ISA
card, I'm now using a Linksys LNEPCI2 card (it's not quite as ancient).

From the table on this MSDN page, "Choosing an Ethernet Card for a CEPC" (
http://msdn2.microsoft.com/en-us/library/aa908177.aspx ), I'm now using the
3rd card listed in that table.

Interrupt Mode allows my OS image to start up much faster (at least twice as
fast, by my rough estimate) as when I was using Polling Mode. I think it's
because of the onslaught of debug messages that require ACK packets. So, it
really was worth getting KITL running in Interrupt Mode. And the Target
Control and Remote Tools features run waaaay faster than they did in
Interrupt Mode.

Hint: If you are switching from an ISA card (which has a fixed IRQ and I/O
port range) to a PCI card, remember to edit your Autoexec.bat on your CEPC
and change to NET_IRQ=0 and NET_IOBASE=0 so that the LOADCEPC program can
auto-detect the PCI board's settings. And if you had edited the kitl_x86.c
file to force KITL to use POLLING mode, be sure to remove that modification.

For those who are interested, here are some details of my debugging of the
interrupt problem with the original (polling-only) network card:

I spent a couple of fruitless hours of getting familiar with the
C:\WINCE600\private\winceos\COREOS\nk\kitl\ethdbg.c code and studying the
debug messages that were sent via the serial port. I could see that the
KITL IST (named "KITLInterruptThread") was stuck waiting for an interrupt
event to occur (which would indicate a LAN packet had arrived), and
apparently it never did, even though network packets destined for my CEPC
were flying around my LAN. Before I started soldering wires onto the net
card so I could watch for IRQs using a logic analyzer, I decided to try a
different network card. And what do you know...it started working!

So maybe IRQ 10 was causing some trouble, or maybe there's a bug in the KITL
code that causes the interrupt to not be seen. (By the way, I was using IRQ
10 and I/O port 0x300 because I had previously connected (and successfully
used) the ISA card on a Win2K machine, and saw those settings in the
Resources section of Device Manager for the network card.)

Hopefully this info will help out others who are seeing this same type of
problem.

-DK

David
Tue Nov 13 12:48:35 PST 2007

I forgot to mention that another drawback to running KITL in Polling Mode is
a degradation in system performance during debugging. While in Polling
Mode, KITL apparently waits in a loop (I guess it's polling for packets) for
a half-second or so, about every 5 seconds, which causes a noticeable "hang"
of the target. For example, if you're moving the mouse around, the mouse
pointer will suddenly stop moving, then after a half-second or so it will
resume moving. Getting KITL to work in Interrupt Mode eliminated this
problem for me.

-DK