Performance statistics

Topics: Mobile Client Software Factory
Jun 7, 2006 at 8:34 PM
originally posted by: Peter_Nowak

I'm still missing the information on hardware used for performance testing. We had this issue already 3 conference calls ago.
Is there a possibility on adding this information for the next drop in "Mobile Client Software Factory - RTP.DOC"?
Moreover the information on the amount of test data would be interesting.
Jun 8, 2006 at 10:11 AM
originally posted by: m_a_madero

Could you also publish the results in performance, because we only have the info about test pass, but it would be interesting to know about
StartUp times with / without CAB
SQLCeReplication versus Subscription Manager
System.Data.SqlServerCe versus DataAccess
StartUp time with Controls Versus Dynamic Resolution Control

It's interesting specially about CAB.

For example, we have been perf testing the ObGen and in the sample application, using the fastBuilder it's 35% slower on the 5.0 emulator and 2% slower on a Dell Axim X30. But once we have the .obgen dll the iterations are in average 25% faster on the emulator and 32% faster on the device and using a factory has no comparission with using a Factory.

This kind of measures allow us to take decissions like, should we use ServiceDependencys in this particular case, would it be better if we build the object inline or should we we build a Factory instead of using the CAB and OB approach.
Jun 8, 2006 at 11:51 AM
originally posted by: dcazzulino

The approach I've been working for the last week regarding optimizations to CAB and OB is to pre-gen code at compile-time, like the approach you already saw in obgen.

obgen is a generator of IPolicyProvider implementations that basically hardcode the entire set of policies that apply to a given type to build. The policies are also optimized, so that the creation policy actually does a new() on the type it's providing the policy too. All dependencies are also optimized for setting and retrieval.

cabgen (coming in the next drop) is a little bit different at the moment, but uses the same approach. It generates a module metadata provider, that gives the (modified) ModuleLoaderService all of the information that was previously retrieved from attributes and reflection, like which are the module initializers, the work item extensions, etc.

As you can see, both tools make runtime reflection just vanish. The only reflection-like behavior is trying to load the providers. If they are not found, we fallback to reflection-based behavior. I think this is a very powerfull complement to traditional attribute-based frameworks, as it provides comparable performance to non reflection-based code, yet preserves the big benefits in usability that you get from attributes.
Jun 8, 2006 at 3:02 PM
originally posted by: carlpf

We used HP Ipaq 2400 520Mhz 64 mb of RAM. Now specifically what test data are you referring. I can send you any specifics about testing metrics and approach.
Jun 8, 2006 at 4:41 PM
originally posted by: PerVonge

We know performance is key for the MCSF and our expert advisory board has been crystal clear - thank you!

In general p&p try to expose the approach and guidelines to establish and meet performance objectives; we apply the same approach for the code assets we ship including MCSF; we do not provide specific numbers.
Jun 9, 2006 at 11:05 AM
originally posted by: m_a_madero

Thanks Daniel, we started something like ObGen, because we tought the MCSF team wasn't doing anything like this.

I tried to use the ObGen, but it's have some troubles using it on other projects, you can see that in the RI.
Probably we aren't using it right, anyway, I will raise a bug to give you more info.
Jun 9, 2006 at 11:06 AM
originally posted by: m_a_madero

Carl we are mainly interested in the perf date of using CAB v.s. not using CAB. For example startup time.
Jun 21, 2006 at 5:38 AM
originally posted by: dcazzulino

Could you add more information on the issues you're running with ObGen?
It basically taks the built-in strategies (property, method, ctor) and generates code that optimizes them to the point that they more or less equal the performance of directly setting a property value in your code, or creating the object, or calling a method.

I personally used the RI as a testbed for it, so I'm curious about the problems you faced. Any feedback will be appreciated.