First group of implementation is done, ODP.NET. I will now start working on the next and last group, using a staging table in oracle. ODP.NET has array insert, regular insert, associative insert. Staging table has bulk insert, external table and SQL*Loader.

Testing for ODP.NET is done: row by row w/o join, row by row w/ join, bulk w/o join, bulk w/ join. Lots of work has been done on the query to get the next and previous value and average from the ten latest values. Since in 99% of the cases we only need the absolute latest value, we probably can do optimizations of this by saving the latest recorded value. The query now requires a full table scan. By limiting the date to maybe 7 days in the past the query would be quicker, and certainly with using partitioning.

Now I use two array sizes, one for testing and one for loading. I have to confirm that the program still works due to this. I will also need to make sure I can determine the commit rate. The commit can be done after each insert, at the end of the entire load or after a certain amount of rows. I will probably go with making a commit after the entire load. If something goes wrong, you can restart the entire load after a rollback. If some rows already are commited we need to restart the process from that point which might be cumbersome. The DB writer and log writer which writes dirty data blocks and the redo data also might be run less frequently. The redo log is also a place to watch out for contention since we need to wait for the redo log to be written to all logs. These logs need to be used for all processes that inserts as well.

Since Oracle uses read consistency (writers and readers don’t block each other) and will write undo data, a frequent commit might slow down the process if a process wants to read from the dirty blocks. The reading process will have to go through the undo data to get the concistency.

A marterialized view might speed up the query to get the static data for testing as well.

Running program locally removes ORA exceptions mentioned in the previous post. Results so far are pretty dissapointing, 100-200 rows processed per second fastest result so far.

Seems like the inserts are taking a lot of time due to db sequential read waits. This is probably due to needing updating indexes. Will probably go from a surrogate key to a natural key to remove an index. This also removes the need for sequences. It’s important to note that a natural key needs to uniquely define a row and preferreably never change. The data model requires a unique index on the natural key anyway so it might be better to skip the surrogate key.

For updating meter values I will probably use RowId instead of the primary key to locate the row. RowIds are the fastest way to access a row since Oracle can deduce in which block the row recides instantly. RowIds can be dangerous to use though depending on circumstances. The RowId of a row can change due to certain events that makes the row change physical place, e.g. reorganizing a table, moving a row between partitions etc. The RowId will be faster then using the primary key since we can skip using the index.

First test has been implemented, which fetches single rows all the time. I am now doing tests which fetches all required rows at the same time.

Inserts:

Besides I/O waits (which is hard to get around), the LogWriter takes a bit of waits (I/O mostly) and also SQL*Net message from client waits. I don’t know if this is because of the network or something else like bad fetch size / packet size. That the data does not fit into one packet.

There are weird behavior in array insert when setting array count to > 2000. The app throws various ORA exceptions from time to time. With a bind count of 1000 it never seems to be any trouble. Need to try this locally and see if the program still exists.

Selects:

For the tests which needs to fetch all rows at the same time the optimizer does not know the cardinality (how many rows there are) in the in parameter tables. This means that it cannot draw certain conclusions in deciding the optimal execution path. The cardinality hint enables us to give the optimizer a hint. According to traces it still seems that the optimizer does a full table scan of tables for in parameter tables which have a pretty low cardinality (>100) where it is more efficient to do a index range scan. I do not know why it draws to this conclusion. This means that it might be necessary to have two procedures where one forces an index range scan (using low cardinality hint) and one where it uses a full table scan. Then it would be necessary to test where the threshold is for when your in parameter tables have enough rows for a full table scan to be beneficial and vice versa. This threshold might also change as the tables in the database grows. Using a 10053 trace you might see why the optimizer chooses the full table scan but I did not get the 10053 trace to work.

Passing in two arrays joining them in the SQL or one custom type array which contain two or more seperate values does not seem to matter.

I have started to implement the different controls that need to be checked on the measurement data. With the test-scenario which will take place on the .NET platform I have done the following alternatives:

Single fetch for each row. This can be used if it is decided to check each row individually. This means that for each row, we go to the database to get the data we need to perform the constraint checks.

Multiple fetch for each table. This will fetch the data needed for several rows, but still make a separate database call for each table in which the information lies. Each stored procedure returns a ref cursor.

Multiple fetch for all tables. This combines the approach above but only takes one database call, minimizing database round trips. Here we have one stored procedure that returns several ref cursors. This stored procedure internally just calls the stored procedure in the prior approach but returns them simultaneously.

The results returned from the above are hashed. Hopefully this will make it possible to filter out data already retrieved in a prior fetch, making in unnecessary to retrieve it again.

Multiple fetch with joined data. This uses a single SQL command, prejoining all the data into one row and returns a ref cursor. This minimizes the amount of SQL executed in the database and will probably be the most efficient approach.

After the controls in .NET have been implemented, it’s time to implement the staging table alternative.

Other things to consider: cardinality hint on the nested tables, apparently the optimizer don’t have statistics on these and will perform a full table scan. The cardinality hint gives the optimizer a clue on how many rows it can expect the nested table to contain which might result in it using a different strategy then a table scan.

Also, one query is using IS NULL on one column, which is very degrading to performance since it cannot use indexes (null values are not indexed). Therefor building a specific index containing null values might be good.

/Karl Trumstedt

When the server was back up, I have been coding more on the general structure of the infrastructure. It is starting to form but still needs work in order to be finished. I have developed three larger scenarios that I will test and these in turn contain a few smaller variations.

What needs to be done right now is to complete the work on the framework, among other things incorporate the oracle measuring tools into the framework.

We are as well starting to define the different controls and tests that are to be made on the test data.

I will also need to continue my essay writing with more implementation details. So far I have only written the background chapter.

/Karl Trumstedt

I have instead spent the day re-designing the access layer a bit and finishing a class for saving performance measurement data to the database (still not tested). This will make it possible to save execution time of different methods. Might take the time to code a simple web page displaying this data.

I will now begin looking on a design for the testing of the real measurement data and will finish up the error logging section of the project.

I would estimate that I have written half of the theory section for the report.

I also attended a meeting with my thesis group where we discussed our projects with Stefan Arnborg.

Pro ODP.NET for Oracle 11G – About Bulkcopy and Associative arrays for insertion.
Oracle PL/SQL – Create procedures which accepts arrays and inserts using FORALL.

Work:
I have spent some time configuring my inserts so they use an automatically incremented value for ID instead of an already given value. This is done with an Oracle sequence and in the INSERT SQL statement use <sequence>.nextval. There seems to be a small performance hit (which is expected) in using this. The question is if this approach will lead to problems using parallel insertions or direct path insert using SQL*Loader and such. It seems to be possible, the question is how much performance will degrade and if there can be duplicate ids. A sequence in Oracle can specify a cache, which caches a number of sequence values. This will improve performance since Oracle does not need to check a system table for the next sequence number. A trade off is that if the database goes down, you will loose the cached sequence numbers.

I have not found any notes on if ODP.NETs bulkcopy command can utilize these sequences.

I have implemented a test-version of inserts using associative arrays using an Oracle stored procedure (which uses FORALL) and ODP.NET. My initial impression is that performance is very good, but not as good as Array  Binding in ODP.NET. According to Pro ODP.NET for ORacle 11g, this is because associative arrays pass down the insert values to the PL/SQL engine which executes the inserts, but Array Binding goes directly to the SQL engine. The PL/SQL engine sends each insert statement one at a time to the SQL engine if using a regular FOR-loop. This results in a lot of context switches between the two engines (Oracle PL/SQL Programming) degrading performance. We can minimize these context switches by using the FORALL statement, which bulks commands and sends them to the SQL engine as a batch.

I have made some work charts describing the work plan and different approaches to test.

Also tried to get the .NET performance counters working for ODP.NET but I didn’t get it to work on either the server or development computer.

/Karl Trumstedt

Fun Oracle fact discovered today: Initrans defined in a table specifies how many sessions that may access the table using the same data block. Freelists are lists of data blocks under the high water mark that can be used for insertions. Multiple freelists allows several sessions to insert data into the database with minimized contention.

I have set up a simple example table in the database, and have a python script that generates a csv file for this table. I have tested a csv reader for .NET that is supposed to be very fast, but I havn’t had a chance to check the source code yet.

I have tried to insert 20000 rows into the example table with regular slow insert using bind variables. Mission accomplished but performance was very slow (expected). I also tried to insert using Array Binding, this was several factors faster (20000 rows under 1 second) but this method failed when trying to bind very large arrays (2 million) insertion failed. Will try different values ranging from 30000 to 60000 to see performance differences.