Long transaction commits.

May 21, 2013 at 11:29 PM
Hi Blaze!

I've noticed two things.
In one transaction i was writing:
  • 8364 x byte[36] structures.
  • 1081 x byte[44] structures.
  • 1122 x byte[8] (date entries).
  • Approximately 8364 nested tables.
All nested tables had Technical_SetTable_OverwriteIsNotAllowed() set.
When I transaction.commit ()-ed
Transaction commit took 958346ms (958 seconds).

When I was observing what was going on was mostly writing small chunks to the .rol file.
610716 bytes - 10000018.rol as You can see rol file is not that big, but it was writing to it
very slowly.


When I attempted this the first time I stopped it at like 200 seconds of work. Restarted the
program. It was taking a lot of time to boot dbreeze. Probably working on .rol file.

I wonder why so many small .rol writes. And such a long time to process rol files.
May 22, 2013 at 7:55 AM
Publish your code.
May 22, 2013 at 8:00 AM
But I think I know where the problem is.
When we have more then 50-100 tables in one transaction, probably Technical_SetTable_OverwriteIsNotAllowed must be also applied to Transaction Journal.
It's not implemented yet.
Restoration from rollback also seems to be natural for such quantity of tables.
May 22, 2013 at 8:03 AM
Yeah. I am thinking of reducing the number of tables by using compound keys rather.
May 22, 2013 at 8:13 AM
I will need some help with that, I'll contact You later. I bet You know some better ways how to achive what I need.
May 22, 2013 at 8:15 AM
In any case publish your code, may be it's gonna be possible to optimize smth.
May 22, 2013 at 8:26 AM
It's kinda difficult because the code is sparse. I'll try, or I'll try to describe it as best as I can.
May 25, 2013 at 2:23 PM
Edited May 25, 2013 at 2:24 PM
Release 049 solves nested tables commit speed.

This code creates 8000 nested tables, writes 1 kvp to each of them and executes 5 seconds.
 private void testC11()
            System.Diagnostics.Stopwatch sw = new System.Diagnostics.Stopwatch();

            using (var tran = engine.GetTransaction())
                NestedTable nt = null;
                for (int i = 1; i <= 8000; i++)
                    nt = tran.InsertTable<int>("t1", i, 0);
                    nt.Insert<int, int>(1, 1);                  

            Console.WriteLine("Consumed: {0}", sw.ElapsedMilliseconds);
May 25, 2013 at 4:00 PM
That's nice blaze. However i got rid of so many tables. But nonetheless this is a great feature.