How to troubleshoot query rewrites

1. Run the script $ORACLE_HOME/rdbms/admin/utlxrw.sql logged in as the schema who owns the materialized view.

2. Analyze the query by logging on as the schema owning the materialized view and execute:

set serveroutput on

begin
      DBMS_MVIEW.EXPLAIN_REWRITE('select dato, max(sekvensnummer)'||
            'from DATE_TO_SEQNUM where dato <= :B1 group by dato','SEQ_AGGREGATED','x');
end;
/

where the first argument is the one you expect to use query rewrite, and the second argument is the name of the materialized view. 3. The potential reasons for the failed rewriting can be found in the table REWRITE_TABLE:

select message from rewrite_table;

What are Index-organized tables and how are they created?

An example of creating an IOT table from my own experience:

CREATE TABLE DATE_TO_SEQNUM (
    DATO DATE, 
    SEQNUM NUMBER(19) NOT NULL,
CONSTRAINT PK_DTS_IOT PRIMARY KEY (DATO,SEQNUM)
)
ORGANIZATION INDEX
TABLESPACE USERS;

If you expect a lot of repeating entries in the IOT, you can use index compression, like this:

CREATE TABLE DATE_TO_SEQNUM (
    DATO DATE, 
    SEQNUM NUMBER(19) NOT NULL,
CONSTRAINT PK_DTS_IOT PRIMARY KEY (DATO,SEQNUM)
)
ORGANIZATION INDEX
TABLESPACE USERS 
COMPRESS;

In my case, the IOT was really this simple. There are some other important directives when creating an IOT, too, which may be applicable in other cases. They are the use of OVERFLOW partition, the INCLUDING keyword, and the PCTTHRESHOLD-clause. I will come back to them when I have had a chance to put them into practice. In the mean time, read about them here.

Here are some statements from the Oracle documentation which defines what an IOT is, and the potential benefits the can provide:

* Index-organized tables are tables stored in an index structure.

* In an index-organized table, rows are stored in an index defined on the primary key for the table. Each index entry in the B-tree also stores the non-key column values. Thus, the index is the data, and the data is the index.

* Index-organized tables provide faster access to table rows by primary key or a valid prefix of the key. The presence of non-key columns of a row in the leaf block avoids an additional data block I/O.

* Index-organized tables are useful when related pieces of data must be stored together or data must be physically stored in a specific order.


Sources: Oracle Database 12.2 documentation

How to create a LIST-RANGE sub-partitioned table in PostgreSQL

Create the table:

CREATE TABLE orders(
  id            integer,
  country_code  VARCHAR(5),
  customer_id   integer,
  order_date    DATE,
  order_total   numeric(8,2)
)
PARTITION BY LIST (country_code);

Create partition for your table. I have defined two. Define it so that it is ready to be further diveded into sub-partitions:

CREATE TABLE orders_NO
 PARTITION OF orders 
 FOR VALUES IN ('no') 
 PARTITION BY RANGE (order_total);

CREATE TABLE orders_SE
 PARTITION OF orders 
 FOR VALUES IN ('se') 
 PARTITION BY RANGE (order_total);

Finally, create your sub-partitions and set their boundaries:

CREATE TABLE small_orders_no
 PARTITION OF orders_NO FOR VALUES FROM (0) TO (2000);

CREATE TABLE medium_orders_no
 PARTITION OF orders_NO FOR VALUES FROM (2001) TO (4000);

CREATE TABLE large_orders_no
 PARTITION OF orders_NO FOR VALUES FROM (4001) to (maxvalue);

CREATE TABLE small_orders_se
PARTITION OF orders_SE FOR VALUES FROM (0) TO (2000);

CREATE TABLE medium_orders_se
 PARTITION OF orders_SE FOR VALUES FROM (2001) TO (4000);

CREATE TABLE large_orders_se
 PARTITION OF orders_SE FOR VALUES FROM (4001) to (maxvalue);

How to avoid ERROR: no partition of relation ... found for row in PostgreSQL

Building on my previous post, I would now like to insert the following rows into my LIST partitioned table:

INSERT INTO ORDERS values (1,'NO',100,'01.02.2018',1000);
INSERT INTO ORDERS values (2,'SE',101,'01.03.2018',1200);

So one rows that I expect to go into the partition for Norway, and another row for the partition for Sweden.

I execute the following:

INSERT INTO ORDERS values (1,'NO',100,'01.02.2018',1000);

which results in an error:

Error: ERROR: no partition of relation "orders" found for row
  Detail: Partition key of the failing row contains (country_code) = (NO).

Contrary to Oracle, with PostgreSQL you need to explicitly create your partition first, like this:

CREATE TABLE orders_NO
 PARTITION OF orders FOR VALUES IN ('NO');

After this is done, PostgreSQL will accept your INSERT-statement.

You can also enforce contraints at the same time, if desirable:

CREATE TABLE orders_NO
 PARTITION OF orders (
 CONSTRAINT orders_id_nn CHECK (id is not null)
) FOR VALUES IN ('NO');

How to create a LIST-partitioned table with a primary key in PostgreSQL

In my experiements with a new RDBMS, I have tried to replicate what I am familiar with from Oracle, to Postgres.

Postgres supports partitioning: RANGE, LIST and HASH.

Here are some findings regarding LIST partitioning, which is extensively used in Oracle by my customers:

To create a list partitioned table in Postgres:

CREATE TABLE orders(
  order_id      integer,
  country_code  VARCHAR(5),
  customer_id   integer,
  order_date    DATE,
  order_total   numeric(8,2)
)
PARTITION BY LIST (country_code);

Add the desired partitions:

CREATE TABLE orders_NO
 PARTITION OF orders FOR VALUES IN ('NO');

CREATE TABLE orders_SE
 PARTITION OF orders FOR VALUES IN ('SE');

When working with Oracle, the equivalent table in Oracle could have been defined with a primary key on the column order_id:

CREATE TABLE orders(
  order_id      number primary key,
  country_code  varchar2(5),
  customer_id   number,
  order_date    date,
  order_total   number(8,2)
)
PARTITION BY LIST (country_code);

With PostgreSQL however, doing so will result in an error:

Error: ERROR: insufficient columns in PRIMARY KEY constraint definition Detail: PRIMARY KEY constraint on table "orders" lacks column "country_code" which is part of the partition key

If the primary key is changed to use the partitioning column, you'll get a valid primary key:

CREATE TABLE orders(
  id            integer,
  country_code  VARCHAR(5) primary key,
  customer_id   integer,
  order_date    DATE,
  order_total   numeric(8,2)
)
PARTITION BY LIST (country_code);

But this is not what I want, since my table is designed to store multiple values for each country.

To create a partitioned table including a valid primary key:

CREATE TABLE orders(
  id            integer,
  country_code  VARCHAR(5),
  customer_id   integer,
  order_date    DATE,
  order_total   numeric(8,2),
PRIMARY KEY (id,country_code)
)
PARTITION BY LIST (country_code);

As long as the primary key includes the partition key, the syntax is accepted by the postgres server.
This is different from Oracle, where you define the primary key independently of the partition key.

As with Oracle, a primary key constraint will automatically create a unique B-tree index.

Note that

PRIMARY KEY (id,country_code)

can alternatively be replaced by

UNIQUE (id,country_code)

The result is the same, either way.

How to connect to a postgres database using jdbc

Using SquirrelSQL I connected to a postgres database as can be seen below:




The entire connection string is

jdbc:postgresql://mypostgresserver01:5432/vegdb01

How to create a multicolumn list-partitioned table in Oracle 12.2

From Oracle 12.2, you can LIST partition a table using multiple columns:

CREATE TABLE regional_orders(
  id            NUMBER,
  country_code  VARCHAR2(5),
  region_code   VARCHAR2(20),
  customer_id   NUMBER,
  order_date    DATE,
  order_total   NUMBER(8,2),
  CONSTRAINT regional_orders_pk PRIMARY KEY (id)
)
PARTITION BY LIST (country_code,region_code) 
(
                PARTITION NORWAY_SOUTH_WEST VALUES (
                                        ( 'NOR','AUST-AGDER'),
                                        ( 'NOR','VEST-AGDER'),
                                        ( 'NOR','ROGALAND')
                                         ),
                PARTITION NORWAY_WEST VALUES (
                                        ( 'NOR','HORDALAND'),
                                        ( 'NOR','SOGN OG FJORDANDE'),
                                        ( 'NOR','MØRE OG ROMSDAL')
                                         ),
                PARTITION NORWAY_CENTRAL VALUES (
                                        ( 'NOR','TRØNDELAG')
                                         ),
                PARTITION NORWAY_EAST VALUES (
                                        ( 'NOR','BUSKERUD'),
                                        ( 'NOR','HEDMARK'),
                                        ( 'NOR','TELEMARK')
                                         )
);

Incoming data that has any of the combinations below, will enter the partition NORWAY_SOUTH_WEST

'NOR','AUST-AGDER'
'NOR','VEST-AGDER'
'NOR','ROGALAND'

This functionality removes the need for LIST-LIST subpartitioning, as far as I can see.


Some queries to verify that the partitions exist:

select  partition_name,partition_position,tablespace_name
from user_tab_partitions 
where table_name='REGIONAL_ORDERS';

PARTITION_NAME	PARTITION_POSITION	TABLESPACE_NAME
NORWAY_SOUTH_WEST	1	USERS
NORWAY_WEST	2	USERS
NORWAY_CENTRAL	3	USERS
NORWAY_EAST	4	USERS

SELECT PARTITIONING_TYPE,PARTITION_COUNT,PARTITIONING_KEY_COUNT 
FROM USER_PART_TABLES 
where table_name='REGIONAL_ORDERS';

PARTITIONING_TYPE	PARTITION_COUNT	PARTITIONING_KEY_COUNT
LIST	4	2

SELECT column_name,column_position 
FROM USER_PART_KEY_COLUMNS
where name='REGIONAL_ORDERS';

COLUMN_NAME	COLUMN_POSITION
COUNTRY_CODE	1
REGION_CODE	2

Let's insert a row:

insert into regional_orders
values (1,'NOR','ROGALAND',3344,'04.01.2019', 1000);
commit;

Analyze the table:

exec DBMS_STATS.GATHER_TABLE_STATS (OwnName => 'SCOTT',TabName => 'REGIONAL_ORDERS');

Verify:

select partition_name, num_rows
from user_tab_partitions 
where table_name='REGIONAL_ORDERS';

PARTITION_NAME	NUM_ROWS
NORWAY_CENTRAL	0
NORWAY_EAST	0
NORWAY_SOUTH_WEST	1
NORWAY_WEST	0

Insert with a different value:

insert into regional_orders
values (2,'NOR','BUSKERUD',3345,'04.01.2019', 1200);

COMMIT;

Analyze again:

exec DBMS_STATS.GATHER_TABLE_STATS (OwnName => 'SCOTT',TabName => 'REGIONAL_ORDERS');

Verify:

select partition_name, num_rows
from user_tab_partitions 
where table_name='REGIONAL_ORDERS';

PARTITION_NAME	NUM_ROWS
NORWAY_CENTRAL	0
NORWAY_EAST	1
NORWAY_SOUTH_WEST	1
NORWAY_WEST	0

If you send in a value that is not covered in your pre-specified list, your statement will fail:

insert into regional_orders
values (3,'NOR','NORDLAND',3346,'04.01.2019', 1100);

ORA-14400: inserted partition key does not map to any partition

The solution to this is to either create your table with a default partition or to use the Oracle 12.2 new feature of automatic list partitioning.

Using a default partition will be as follows:

CREATE TABLE regional_orders(
  id            NUMBER,
  country_code  VARCHAR2(5),
  region_code   VARCHAR2(20),
  customer_id   NUMBER,
  order_date    DATE,
  order_total   NUMBER(8,2),
  CONSTRAINT regional_orders_pk PRIMARY KEY (id)
)
PARTITION BY LIST (country_code,region_code) 
                (       
                PARTITION NORWAY_SOUTH_WEST VALUES (
                                        ( 'NOR','AUST-AGDER'),
                                        ( 'NOR','VEST-AGDER'),
                                        ( 'NOR','ROGALAND')
                                         ),
                PARTITION NORWAY_WEST VALUES (
                                        ( 'NOR','HORDALAND'),
                                        ( 'NOR','SOGN OG FJORDANDE'),
                                        ( 'NOR','MØRE OG ROMSDAL')
                                         ),
                PARTITION NORWAY_CENTRAL VALUES (
                                        ( 'NOR','TRØNDELAG')
                                         ),
                PARTITION NORWAY_EAST VALUES (
                                        ( 'NOR','BUSKERUD'),
                                        ( 'NOR','HEDMARK'),
                                        ( 'NOR','TELEMARK')
                                         ),
                PARTITION NORWAY_REST VALUES (DEFAULT) 
                )
;

Using the AUTOMATIC keyword will be as easy as adding the keyword AUTOMATIC to your partitioning-clause and to remove the specification for your overflow partition:

CREATE TABLE regional_orders(
  id            NUMBER,
  country_code  VARCHAR2(5),
  region_code   VARCHAR2(20),
  customer_id   NUMBER,
  order_date    DATE,
  order_total   NUMBER(8,2),
  CONSTRAINT regional_orders_pk PRIMARY KEY (id)
)
PARTITION BY LIST (country_code,region_code) AUTOMATIC
                (       
                PARTITION NORWAY_SOUTH_WEST VALUES (
                                        ( 'NOR','AUST-AGDER'),
                                        ( 'NOR','VEST-AGDER'),
                                        ( 'NOR','ROGALAND')
                                         ),
                PARTITION NORWAY_WEST VALUES (
                                        ( 'NOR','HORDALAND'),
                                        ( 'NOR','SOGN OG FJORDANDE'),
                                        ( 'NOR','MØRE OG ROMSDAL')
                                         ),
                PARTITION NORWAY_CENTRAL VALUES (
                                        ( 'NOR','TRØNDELAG')
                                         ),
                PARTITION NORWAY_EAST VALUES (
                                        ( 'NOR','BUSKERUD'),
                                        ( 'NOR','HEDMARK'),
                                        ( 'NOR','TELEMARK')
                                         )
                )
;

New values will be added automatically and given system-generated partition names.

There is an obvious trade-off here: if you use automation, you will not be able to give your partition logical names and place values logically belong together, in the same partition.

For example, the values

'NOR','NORDLAND'

may logically belong to the same partition as the values

'NOR','TROMS'
and
'NOR','FINNMARK'

If I knew such a value may be sent at the time of table creation, I would of course create another partition, like this:

PARTITION NORWAY_NORTH VALUES (
                                        ( 'NOR','NORDLAND'),
                                        ( 'NOR','TROMS'),
                                        ( 'NOR','FINNMARK')
                                         )

If you use a default partition, you will always be able to save your incoming rows, and you have full controll over your partition names and their values.
But new rows end up in an overflow partition together with any other record that cannot find its way into a properly named and configured partition.
You may need to split your default partition later, to satisify your business needs or to reduce the size of your default partition.