Running from RAM, demand paging

Hi. I'm trying to run everything from RAM, all the time. I seem to be
having problems.

On startup, if a compact flash is present, and it contains an, .exe,
we do a CreateFile on it. Windows CE nicely creates a copy in RAM, in
the object store, and ignores the one in flash. We discovered that
when we do this, we get a *dramatic* performance improvement. A device
driver ran twice as fast. I'm assuming what we are doing is faster
because we are running from RAM. I'm assuming that otherwise Windows
CE loads + unloads the code from flash to RAM as necessary.

I would like to make all our code load from flash into RAM on startup
(or on first use) and stay there running in RAM forever.

I tried setting ROMFLAGS = 0x0001 in config.bib, which I think is
supposed to run everything in kernel mode and disable demand paging. I
cleaned everything, deleted CESysGen and release directories, and
rebuilt... but I got no increase in speed, at all.

Any suggestions?

Thanks

Andy

Silver
Thu Jul 12 20:21:26 CDT 2007

I'm not quite sure what the CreateFile on an Exe is all about, did you mean
CreateProcess?

You need to change your config.bib to use RAM addresses for the ROMIMAGE
section (and any other XIP sections that you have) then have your bootloader
parse the nk.bin into ram at boot time then execute in ram - when I say
parse this may be as simple as a copy loop or you could parse the TOC if you
were feeling adventerous.

Disabling demand paging won't do what you want, it is used to execute
application in RAM rather then from flash/ROM so you just forced the kernel
to always run from ROM.


Geoff
--

Andrew
Fri Jul 13 08:19:52 CDT 2007

Thanks! We copy the code to RAM for our other board. I can use that as
a model.

> I'm not quite sure what the CreateFile on an Exe is all about, did you mean
> CreateProcess?

We copy from a compact flash card. So the file already exists in
flash. On the storage card we find another file with the same name. We
read the copy on the storage card into RAM, do a
CreateFile(CREATE_ALWAYS) over the file in flash, then write the data
in RAM to the newly created file.

> You need to change your config.bib to use RAM addresses for the ROMIMAGE
> section (and any other XIP sections that you have) then have your bootloader
> parse the nk.bin into ram at boot time then execute in ram - when I say
> parse this may be as simple as a copy loop or you could parse the TOC if you
> were feeling adventerous.
>
We burn our image into flash as just a big array, so I think we should
probably just be able to copy it to RAM that way.

> Disabling demand paging won't do what you want, it is used to execute
> application in RAM rather then from flash/ROM so you just forced the kernel
> to always run from ROM.

Oh, I definitely don't want to do that then. It sounds like it
disables not only demand paging, but any paging.

>
> Geoff
> --

Dean
Fri Jul 13 09:16:11 CDT 2007

> Disabling demand paging won't do what you want, it is used to execute
> application in RAM rather then from flash/ROM so you just forced the
> kernel to always run from ROM.
>

Maybe I misunderstood your comment, but disabling demand paging means that
the page table entries necessary to execute the code sections of an exe/dll
will be fully set up when the exe/dll loads and physical RAM pages will be
allocated. Normally an exception occurs on first access for every page, and
the OS maps memory in on demand. It shouldn't have anything to do with
running from RAM/ROM.

--
Dean Ramsier - eMVP
BSQUARE Corporation

Andrew
Fri Jul 13 09:39:10 CDT 2007

>
> Maybe I misunderstood your comment, but disabling demand paging means that
> the page table entries necessary to execute the code sections of an exe/dll
> will be fully set up when the exe/dll loads and physical RAM pages will be
> allocated. Normally an exception occurs on first access for every page, and
> the OS maps memory in on demand. It shouldn't have anything to do with
> running from RAM/ROM.
>

We don't observe any speedup from this flag. But if we "CreateFile" a
device driver over a file already in ROM, thereby (I think) creating a
RAM copy, things get much faster. Does that make any sense to you?

> --
> Dean Ramsier - eMVP
> BSQUARE Corporation

Dean
Fri Jul 13 13:19:44 CDT 2007

My comment was regarding Geoff's explanation of the flag.

Yes, if you move the driver into RAM, then it will likely be faster (RAM is
much faster than flash). It's a simple tradeoff - you have a limited amount
of ram and you can use it to execute the image, or you can use it for
program memory/storage space. Executing directly from flash is a feature of
CE that allows best use of limited resources, but it's a feature that might
not be important to you.

If you have enough RAM, then put the whole image into RAM as Geoff
suggested. If you're trying to preserve RAM, then you can identify critical
components and force them into RAM individually by setting the C
(compressed) bit in the bib file.

--
Dean Ramsier - eMVP
BSQUARE Corporation


"Andrew Schweitzer" <anschweitzer@gmail.com> wrote in message
news:1184337550.425506.20150@n60g2000hse.googlegroups.com...
> >
>> Maybe I misunderstood your comment, but disabling demand paging means
>> that
>> the page table entries necessary to execute the code sections of an
>> exe/dll
>> will be fully set up when the exe/dll loads and physical RAM pages will
>> be
>> allocated. Normally an exception occurs on first access for every page,
>> and
>> the OS maps memory in on demand. It shouldn't have anything to do with
>> running from RAM/ROM.
>>
>
> We don't observe any speedup from this flag. But if we "CreateFile" a
> device driver over a file already in ROM, thereby (I think) creating a
> RAM copy, things get much faster. Does that make any sense to you?
>
>> --
>> Dean Ramsier - eMVP
>> BSQUARE Corporation
>
>

Andrew
Wed Jul 18 12:12:38 CDT 2007

On Jul 13, 2:19 pm, "Dean Ramsier" <ramsiernos...@nospam.com> wrote:

> If you have enough RAM, then put the whole image into RAM as Geoff
> suggested. If you're trying to preserve RAM, then you can identify critical
> components and force them into RAM individually by setting the C
> (compressed) bit in the bib file.
>

Thanks, I will be trying this and possibly loading the whole image to
RAM shortly.

Do you know what the Disable Demand Paging actually does, if it
doesn't do the same thing as the C bit does?

My impression was that CE always ran in RAM unless you forced
something to be XIP in the bib file. I seem to have gotten the
impression from some doc somewhere that it loaded the code it needed
(not the whole image of an executeable) into RAM when "necessary" and
might unload it when it determined it was now "unecessary". Based on
this I assumed Disable Demand Paging forced it to load the entire
executable into RAM when the executable started and only unload it
after it stopped. Can you correct these impressions?

Thanks,

Andy

Dean
Wed Jul 18 16:11:21 CDT 2007

'C' isn't a bit, it is a flag that directs the romimage tool to compress a
particular module. The module must then be decompressed in order to run,
and the decompressed version will of necessity be in RAM. By default,
components listed in the MODULES section are not compressed so they can
execute in place (which might be RAM or ROM). Setting the C flag on a
component in the MODULES section is a convenient shortcut to ensure a
particular component executes from RAM regardless of whether the entire
image is running from RAM or ROM. Modules that have the C flag set will run
from RAM, but only due to a side effect.

Note that components listed in the FILES section are compressed by default.
You can force them to be uncompressed for performance reasons by setting the
'U' flag in the bib file. Components in the FILES section will also always
run from RAM, because they must be fixed up by the loader into RAM before
they can run.

The Disable Demand Paging bit in the ROMFLAGS of config.bib is different.
It will cause the loader to completely allocate all physical RAM pages that
are necessary when the module loads, and lock them so they are not paged
out. If there aren't enough physical pages available the load attempt will
fail. Of course, if the module is XIP from flash (or from RAM) there is no
need to allocate physical pages because the memory that it executes from is
already present. In that case, the demand paging is irrelevant.

So setting the Disable Demand Paging bit doesn't guarantee that a module
runs from RAM, it merely guarantees that if the module needs to be loaded
into RAM it will be fully loaded or not at all.

--
Dean Ramsier - eMVP
BSQUARE Corporation


"Andrew Schweitzer" <anschweitzer@gmail.com> wrote in message
news:1184778758.840747.224420@e9g2000prf.googlegroups.com...
> On Jul 13, 2:19 pm, "Dean Ramsier" <ramsiernos...@nospam.com> wrote:
>
>> If you have enough RAM, then put the whole image into RAM as Geoff
>> suggested. If you're trying to preserve RAM, then you can identify
>> critical
>> components and force them into RAM individually by setting the C
>> (compressed) bit in the bib file.
>>
>
> Thanks, I will be trying this and possibly loading the whole image to
> RAM shortly.
>
> Do you know what the Disable Demand Paging actually does, if it
> doesn't do the same thing as the C bit does?
>
> My impression was that CE always ran in RAM unless you forced
> something to be XIP in the bib file. I seem to have gotten the
> impression from some doc somewhere that it loaded the code it needed
> (not the whole image of an executeable) into RAM when "necessary" and
> might unload it when it determined it was now "unecessary". Based on
> this I assumed Disable Demand Paging forced it to load the entire
> executable into RAM when the executable started and only unload it
> after it stopped. Can you correct these impressions?
>
> Thanks,
>
> Andy
>
>

Andrew
Mon Jul 30 14:00:23 CDT 2007

Thanks for these explanations.

I did some more experiments and determined that:
-->The 'C' flag seemed to have no affect on speed. It didn't use up
any extra memory either.
-->Disable Demand Paging seemed to have no affect on speed. It used up
an extra 1Mb of RAM.
-->Full Kernel Mode improved speed about 10%.
-->Loading files from storage card improved speed about 10%. It used
up an extra 1 - 2 Mb of RAM.

The last two were cumulative, so together they improved speed by about
20%.

Does this make any sense?

You said below that
> [Disable demand paging] will cause the loader to completely allocate all physical RAM pages that
> are necessary when the module loads, and lock them so they are not paged
> out.

...but then...
> So setting the Disable Demand Paging bit doesn't guarantee that a module
> runs from RAM, it merely guarantees that if the module needs to be loaded
> into RAM it will be fully loaded or not at all.

I'm confused about these two statements. They seem to be
contradictory. What am I missing?


Do you know why Full Kernel Mode speeds things up so much? My
understanding is that instructions that would otherwise cause
exceptions will not if the processor is already in supervisor mode, so
we just cut out a lot of exception processing.

Thanks,

Andy


On Jul 18, 5:11 pm, "Dean Ramsier" <ramsiernos...@nospam.com> wrote:
> 'C' isn't a bit, it is a flag that directs the romimage tool to compress a
> particular module. The module must then be decompressed in order to run,
> and the decompressed version will of necessity be in RAM. By default,
> components listed in the MODULES section are not compressed so they can
> execute in place (which might be RAM or ROM). Setting the C flag on a
> component in the MODULES section is a convenient shortcut to ensure a
> particular component executes from RAM regardless of whether the entire
> image is running from RAM or ROM. Modules that have the C flag set will run
> from RAM, but only due to a side effect.
>
> Note that components listed in the FILES section are compressed by default.
> You can force them to be uncompressed for performance reasons by setting the
> 'U' flag in the bib file. Components in the FILES section will also always
> run from RAM, because they must be fixed up by the loader into RAM before
> they can run.
>
> The Disable Demand Paging bit in the ROMFLAGS of config.bib is different.
> It will cause the loader to completely allocate all physical RAM pages that
> are necessary when the module loads, and lock them so they are not paged
> out. If there aren't enough physical pages available the load attempt will
> fail. Of course, if the module is XIP from flash (or from RAM) there is no
> need to allocate physical pages because the memory that it executes from is
> already present. In that case, the demand paging is irrelevant.
>
> So setting the Disable Demand Paging bit doesn't guarantee that a module
> runs from RAM, it merely guarantees that if the module needs to be loaded
> into RAM it will be fully loaded or not at all.
>
> --
> Dean Ramsier - eMVP
> BSQUARE Corporation
>
> "Andrew Schweitzer" <anschweit...@gmail.com> wrote in message
>
> news:1184778758.840747.224420@e9g2000prf.googlegroups.com...
>
> > On Jul 13, 2:19 pm, "Dean Ramsier" <ramsiernos...@nospam.com> wrote:
>
> >> If you have enough RAM, then put the whole image into RAM as Geoff
> >> suggested. If you're trying to preserve RAM, then you can identify
> >> critical
> >> components and force them into RAM individually by setting the C
> >> (compressed) bit in the bib file.
>
> > Thanks, I will be trying this and possibly loading the whole image to
> > RAM shortly.
>
> > Do you know what the Disable Demand Paging actually does, if it
> > doesn't do the same thing as the C bit does?
>
> > My impression was that CE always ran in RAM unless you forced
> > something to be XIP in the bib file. I seem to have gotten the
> > impression from some doc somewhere that it loaded the code it needed
> > (not the whole image of an executeable) into RAM when "necessary" and
> > might unload it when it determined it was now "unecessary". Based on
> > this I assumed Disable Demand Paging forced it to load the entire
> > executable into RAM when the executable started and only unload it
> > after it stopped. Can you correct these impressions?
>
> > Thanks,
>
> > Andy

Dean
Tue Jul 31 08:45:07 CDT 2007

Are your components listed in the FILES section or the MODULES section of
the .bib file? If in the MODULES section, the C flag should cause them to
be decompressed to RAM, which should take up some amount more space. If in
the FILES section, they are already compressed so adding the C flag would
have no effect. I'm not entirely sure how the OS deals with compressed
modules, so I can't say with certainty why you aren't seeing an increase in
speed. But in my experience, setting that flag on a component listed in the
MODULES section in an image that was otherwise running from ROM increased
the performance of that component.

When running from the storage card, you are definitely loading your
components into RAM and executing from RAM. I think this should be
comparable to having the components listed in the bib file under FILES,
although I'm not sure that the paging works the same for components coming
from the image versus external storage.

Disable Demand Paging will definitely use up more RAM because the entire
module is loaded into physical RAM at once, instead of on demand. It will
only increase performance if there had been a lot of demand paging occuring
and those events were disruptive enough to show up in your performance
measurement.

Full Kernel Mode will increase performance for an entirely unrelated reason.
Your applications and drivers will now be running in the supervisory kernel
mode instead of the lower privilege user mode. This gives them the ability
to access memory locations that are marked for kernel mode only (e.g. all
the memory above 0x80000000, the memory in kernel space). Since they are
already running in kernel mode, system calls are quicker because the OS
doesn't have to transition back and forth between user and kernel mode
(which requires additional exception handling). This is traded off with
robustness since the kernel memory space (which includes access to all
physical RAM) is now directly accessible to user applications.

The confusion from the last two statements is due to the fact that a module
can execute directly in place from *ROM* using XIP. If the image is running
from ROM, an uncompressed module listed in the MODULES section will execute
in place in ROM. For the same reason, if an image is running from RAM, an
uncompressed module listed in the MODULES section will execute in place in
RAM. In neither case will demand paging occur - the memory where the code
executes (either RAM or ROM) is already resident as part of the image
itself.

So disabling demand paging does not guarantee that a module runs from RAM,
it just means that demand paging won't occur. It might not occur for other
reasons either, such as when an exe is already executing in place. If you
want to guarantee that code is executing in RAM, you have to ensure that it
is forced there under all conditions.

--
Dean Ramsier - eMVP
BSQUARE Corporation


"Andrew Schweitzer" <anschweitzer@gmail.com> wrote in message
news:1185822023.629458.212630@r34g2000hsd.googlegroups.com...
> Thanks for these explanations.
>
> I did some more experiments and determined that:
> -->The 'C' flag seemed to have no affect on speed. It didn't use up
> any extra memory either.
> -->Disable Demand Paging seemed to have no affect on speed. It used up
> an extra 1Mb of RAM.
> -->Full Kernel Mode improved speed about 10%.
> -->Loading files from storage card improved speed about 10%. It used
> up an extra 1 - 2 Mb of RAM.
>
> The last two were cumulative, so together they improved speed by about
> 20%.
>
> Does this make any sense?
>
> You said below that
>> [Disable demand paging] will cause the loader to completely allocate all
>> physical RAM pages that
>> are necessary when the module loads, and lock them so they are not paged
>> out.
>
> ...but then...
>> So setting the Disable Demand Paging bit doesn't guarantee that a module
>> runs from RAM, it merely guarantees that if the module needs to be loaded
>> into RAM it will be fully loaded or not at all.
>
> I'm confused about these two statements. They seem to be
> contradictory. What am I missing?
>
>
> Do you know why Full Kernel Mode speeds things up so much? My
> understanding is that instructions that would otherwise cause
> exceptions will not if the processor is already in supervisor mode, so
> we just cut out a lot of exception processing.
>
> Thanks,
>
> Andy
>
>
> On Jul 18, 5:11 pm, "Dean Ramsier" <ramsiernos...@nospam.com> wrote:
>> 'C' isn't a bit, it is a flag that directs the romimage tool to compress
>> a
>> particular module. The module must then be decompressed in order to
>> run,
>> and the decompressed version will of necessity be in RAM. By default,
>> components listed in the MODULES section are not compressed so they can
>> execute in place (which might be RAM or ROM). Setting the C flag on a
>> component in the MODULES section is a convenient shortcut to ensure a
>> particular component executes from RAM regardless of whether the entire
>> image is running from RAM or ROM. Modules that have the C flag set will
>> run
>> from RAM, but only due to a side effect.
>>
>> Note that components listed in the FILES section are compressed by
>> default.
>> You can force them to be uncompressed for performance reasons by setting
>> the
>> 'U' flag in the bib file. Components in the FILES section will also
>> always
>> run from RAM, because they must be fixed up by the loader into RAM before
>> they can run.
>>
>> The Disable Demand Paging bit in the ROMFLAGS of config.bib is different.
>> It will cause the loader to completely allocate all physical RAM pages
>> that
>> are necessary when the module loads, and lock them so they are not paged
>> out. If there aren't enough physical pages available the load attempt
>> will
>> fail. Of course, if the module is XIP from flash (or from RAM) there is
>> no
>> need to allocate physical pages because the memory that it executes from
>> is
>> already present. In that case, the demand paging is irrelevant.
>>
>> So setting the Disable Demand Paging bit doesn't guarantee that a module
>> runs from RAM, it merely guarantees that if the module needs to be loaded
>> into RAM it will be fully loaded or not at all.
>>
>> --
>> Dean Ramsier - eMVP
>> BSQUARE Corporation
>>
>> "Andrew Schweitzer" <anschweit...@gmail.com> wrote in message
>>
>> news:1184778758.840747.224420@e9g2000prf.googlegroups.com...
>>
>> > On Jul 13, 2:19 pm, "Dean Ramsier" <ramsiernos...@nospam.com> wrote:
>>
>> >> If you have enough RAM, then put the whole image into RAM as Geoff
>> >> suggested. If you're trying to preserve RAM, then you can identify
>> >> critical
>> >> components and force them into RAM individually by setting the C
>> >> (compressed) bit in the bib file.
>>
>> > Thanks, I will be trying this and possibly loading the whole image to
>> > RAM shortly.
>>
>> > Do you know what the Disable Demand Paging actually does, if it
>> > doesn't do the same thing as the C bit does?
>>
>> > My impression was that CE always ran in RAM unless you forced
>> > something to be XIP in the bib file. I seem to have gotten the
>> > impression from some doc somewhere that it loaded the code it needed
>> > (not the whole image of an executeable) into RAM when "necessary" and
>> > might unload it when it determined it was now "unecessary". Based on
>> > this I assumed Disable Demand Paging forced it to load the entire
>> > executable into RAM when the executable started and only unload it
>> > after it stopped. Can you correct these impressions?
>>
>> > Thanks,
>>
>> > Andy
>
>