Showing posts with label index partitioning. Show all posts
Showing posts with label index partitioning. Show all posts

Wednesday, August 24, 2022

Does PostgreSQL support global indexes on partitioned tables?

PostgreSQL does not support Global Indexes, i.e indexes that spans all partitions in a partitioned table. The documentation states:

Although all partitions must have the same columns as their partitioned parent, partitions may have their own indexes, constraints and default values, distinct from those of other partitions.

and

Create an index on the key column(s), as well as any other indexes you might want, on the partitioned table. ... This automatically creates a matching index on each partition, and any partitions you create or attach later will also have such an index. An index or unique constraint declared on a partitioned table is “virtual” in the same way that the partitioned table is: the actual data is in child indexes on the individual partition tables. A proposal to add such feature seem to have been suggested back in 2019, but currently no implementation exists.

I also found this comparison between Oracle and PostgreSQL that elaborates on the topic a bit more.

Wednesday, July 29, 2020

How to rebuild an index partition

To limit a rebuild to a specific index partition only, use the following syntax:
ALTER INDEX SH.SALES_IDX4 
REBUILD PARTITION SYS_P51490 
TABLESPACE DATA04 ONLINE;
Notice the use of the ONLINE keyward, which will allow DML against the table (and thus update the index). To generate rebuild-statements, one index at a time, I have used the script below. It takes as arguments
  1. The index owner 
  2. The index name 
  3. The new tablespace where you want to place your partitions 
  4. The old tablespace from which you want to move out
set termout off
select 'alter session set nls_language=''american'';' from dual;
set termout on
accept index_owner prompt 'Index owner: '
accept index_name prompt 'Index name: '
accept new_tabspc prompt 'New tablespace: '
accept old_tabspc prompt 'Old tablespace: '

set lines 300
set heading off
set feedback off
set verify off
set echo off
set pages 0
set trimspool on
spool exec_move_part_&&index_owner..&&index_name..sql
select 'alter session set nls_language=''american'';' from dual;

select 'alter session force parallel ddl;' from dual;
select 'set timing on' from dual;
select 'set lines 200' from dual;
select 'set trimspool on' from dual;
select 'spool exec_rebuild_part_&&index_owner..&&index_name..log' from dual;
select 'exec DBMS_APPLICATION_INFO.SET_MODULE(module_name=>''Move'', action_name=>''Rebuild_part_&&index_name'');' from dual;
select 'exec DBMS_APPLICATION_INFO.SET_CLIENT_INFO (client_info=>''Rebuild_part_t_&&index_name'');' from dual;
select 'exec DBMS_SESSION.SET_IDENTIFIER (client_id=>''Rebuild_part_&&index_name''); ' from dual;

select 'prompt moving the index partitions from &&old_tabspc to &&new_tabspc,' from dual;
select 'prompt and setting default attributes for the table ' from dual;
SELECT 'alter index ' || idx.table_owner || '.' ||  idx.index_name || ' rebuild partition ' || idxpart.partition_name || ' TABLESPACE &&new_tabspc ONLINE;'
FROM    dba_indexes idx, dba_ind_partitions idxpart
WHERE   idx.table_owner = '&&index_owner'
and     idxpart.TABLESPACE_NAME = '&&old_tabspc'
AND     idx.index_name = idxpart.index_name
AND     idx.index_name = '&&index_name'
AND     idxpart.subpartition_count=0
ORDER BY idx.table_owner, idx.index_name;

select distinct 'alter index ' ||   IDXPART.INDEX_OWNER || '.' || IDXPART.INDEX_NAME || ' modify default attributes tablespace &&new_tabspc;'
FROM   DBA_IND_PARTITIONS IDXPART JOIN DBA_SEGMENTS S
ON     (IDXPART.PARTITION_NAME = S.PARTITION_NAME )
WHERE  IDXPART.INDEX_NAME='&&index_name'
AND    IDXPART.TABLESPACE_NAME='&&old_tabspc';

select 'exit' from dual;
exit

Run it as a user with access to the data dictionary: 

sqlplus / as sysdba @gen_move_idx_part.sql 

It will generate a new script called exec_move_part_<username>.<index_name>.sql which will perform the actual rebuild: 
sqlplus / as sysdba @exec_move_part_<username>.<index_name>.sql

I also added som calls to dbms_application_info so that the session can be easily identfied in v$session. Remember, sometimes it's desirable to set the PARALLEL degree and the NOLOGGING options during rebuild, to complete it as fast as possible. See this post for more info.

Tuesday, January 8, 2019

How to create a LIST-partitioned table with a primary key in PostgreSQL


In my experiements with a new RDBMS, I have tried to replicate what I am familiar with from Oracle, to Postgres.

Postgres supports partitioning: RANGE, LIST and HASH.

Here are some findings regarding LIST partitioning, which is extensively used in Oracle by my customers:

To create a list partitioned table in Postgres:

CREATE TABLE orders(
  order_id      integer,
  country_code  VARCHAR(5),
  customer_id   integer,
  order_date    DATE,
  order_total   numeric(8,2)
)
PARTITION BY LIST (country_code);

Add the desired partitions:
CREATE TABLE orders_NO
 PARTITION OF orders FOR VALUES IN ('NO');

CREATE TABLE orders_SE
 PARTITION OF orders FOR VALUES IN ('SE');

When working with Oracle, the equivalent table in Oracle could have been defined with a primary key on the column order_id:

CREATE TABLE orders(
  order_id      number primary key,
  country_code  varchar2(5),
  customer_id   number,
  order_date    date,
  order_total   number(8,2)
)
PARTITION BY LIST (country_code);

With PostgreSQL however, doing so will result in an error:
Error: ERROR: insufficient columns in PRIMARY KEY constraint definition
Detail: PRIMARY KEY constraint on table "orders" lacks column "country_code" which is part of the partition key

If the primary key is changed to use the partitioning column, you'll get a valid primary key:
CREATE TABLE orders(
  id            integer,
  country_code  VARCHAR(5) primary key,
  customer_id   integer,
  order_date    DATE,
  order_total   numeric(8,2)
)
PARTITION BY LIST (country_code);

But this is not what I want, since my table is designed to store multiple values for each country.

To create a partitioned table including a valid primary key:
CREATE TABLE orders(
  id            integer,
  country_code  VARCHAR(5),
  customer_id   integer,
  order_date    DATE,
  order_total   numeric(8,2),
PRIMARY KEY (id,country_code)
)
PARTITION BY LIST (country_code);

As long as the primary key includes the partition key, the syntax is accepted by the postgres server.
This is different from Oracle, where you define the primary key independently of the partition key.

As with Oracle, a primary key constraint will automatically create a unique B-tree index.

Note that
PRIMARY KEY (id,country_code)
can alternatively be replaced by
UNIQUE (id,country_code)

The result is the same, either way.

Tuesday, August 1, 2017

How to rebuild an index subpartition



To limit a rebuild to a specific index subpartition only, use the following syntax:

ALTER INDEX SCOTT.SALES_IDX3
  REBUILD SUBPARTITION SALES_IDX3_SP_2017_06 ONLINE
  TABLESPACE DATA2;

Notice the use of the ONLINE keyward, which will allow DML against the table (and thus update the index).

To generate rebuild-statements, one index at a time, I have used the script below.

It takes as arguments
1. The index owner
2. The index name
3. The new tablespace where you want to place your subpartitions
4. The old tablespace from which you want to move out


set termout off
select 'alter session set nls_language=''american'';' from dual;
set termout on
accept index_owner prompt 'Index owner: '
accept index_name prompt 'Index name: '
accept new_tabspc prompt 'New tablespace: '
accept old_tabspc prompt 'Old tablespace: '

set lines 300
set heading off
set feedback off
set verify off
set echo off
set pages 0
set trimspool on
spool exec_move_subpart_&&index_owner..&&index_name..sql
select 'alter session set nls_language=''american'';' from dual;

select 'alter session force parallel ddl;' from dual;
select 'set timing on' from dual;
select 'set lines 200' from dual;
select 'set trimspool on' from dual;
select 'spool exec_rebuild_subpart_&&index_owner..&&index_name..log' from dual;
select 'exec DBMS_APPLICATION_INFO.SET_MODULE(module_name=>''Move'', action_name=>''Rebuild_supbart_&&index_name'');' from dual;
select 'exec DBMS_APPLICATION_INFO.SET_CLIENT_INFO (client_info=>''Rebuild_supbart_t_&&index_name'');' from dual;
select 'exec DBMS_SESSION.SET_IDENTIFIER (client_id=>''Rebuild_supbart_&&index_name''); ' from dual;

select 'prompt moving the index subpartitions from &&old_tabspc to &&new_tabspc,' from dual;
select 'prompt and setting default attributes for the involved partitions. ' from dual;
SELECT 'ALTER INDEX ' ||  IDXSUBPART.INDEX_OWNER || '.' || IDXSUBPART.INDEX_NAME || ' rebuild subpartition ' || IDXSUBPART.SUBPARTITION_NAME || ' tablespace &&new_tabspc ONLINE;'
FROM   DBA_IND_SUBPARTITIONS IDXSUBPART JOIN DBA_SEGMENTS S
ON     (IDXSUBPART.SUBPARTITION_NAME = S.PARTITION_NAME )
WHERE  IDXSUBPART.INDEX_NAME='&&index_name'
AND    IDXSUBPART.TABLESPACE_NAME='&&old_tabspc'
AND    S.SEGMENT_TYPE='INDEX SUBPARTITION'
union
select distinct 'alter index ' ||   IDXSUBPART.INDEX_OWNER || '.' || IDXSUBPART.INDEX_NAME || ' modify default attributes for partition ' || idxsubpart.partition_name || ' tablespace &&new_tabspc;'
FROM   DBA_IND_SUBPARTITIONS IDXSUBPART JOIN DBA_SEGMENTS S
ON     (IDXSUBPART.SUBPARTITION_NAME = S.PARTITION_NAME )
WHERE  IDXSUBPART.INDEX_NAME='&&index_name'
AND    IDXSUBPART.TABLESPACE_NAME='&&old_tabspc'
AND    S.SEGMENT_TYPE='INDEX SUBPARTITION';

select 'exit' from dual;
exit


Run it as a user with access to the data dictionary:
sqlplus / as sysdba @gen_move_idx_subpart.sql

It will generate a new script called exec_move_subpart_<username>.<index_name>.sql which will actually perform the rebuild:
sqlplus / as sysdba @exec_move_subpart_<username>.<index_name>.sql
I also added som calls to dbms_application_info so that the session can be easily found in v$session

Tuesday, January 14, 2014

How to generate a script to rebuild unusable index partitions



set trimspool on
set lines     200
set pages     0
set heading   off
set verify    off
set feedback  off
set echo      off
spool rebuild_stmts.sql
--Rebuild the index partition to the same tablespace:
SELECT 'alter index ' || idx.table_owner || '.' ||  idx.index_name || ' rebuild partition ' || idxpart.partition_name || ';'
FROM   dba_indexes idx, dba_ind_partitions idxpart
WHERE  idx.table_owner = 'SCOTT'
AND    idx.index_name = idxpart.index_name
AND    idxpart.status <> 'USABLE'
ORDER BY idx.table_owner, idx.index_name
/
--For a specific INDEX, reallocating the index partitions to a new tablespace:
SELECT 'alter index ' || idx.table_owner || '.' ||  idx.index_name || ' rebuild partition ' || idxpart.partition_name || ' TABLESPACE MYTABLESPACE;'
FROM    dba_indexes idx, dba_ind_partitions idxpart
WHERE   idx.table_owner = 'SCOTT'
AND     idx.index_name = idxpart.index_name
AND     idx.index_name = 'IDX_01'
AND     idxpart.status <> 'USABLE'
ORDER BY idx.table_owner, idx.index_name
/
--Use sqlplus variables, also reallocating the index partitions to a new tablespace:
SELECT 'alter index ' || idx.table_owner || '.' ||  idx.index_name || ' rebuild partition ' || idxpart.partition_name || ' TABLESPACE MYTABLESPACE;'
FROM    dba_indexes idx, dba_ind_partitions idxpart
WHERE   idx.table_owner = 'SCOTT'
AND     idx.index_name = idxpart.index_name
AND     idx.index_name = '&&index_name'
AND     idxpart.status <> 'USABLE'
ORDER BY idx.table_owner, idx.index_name
/
-- Rebuild index subpartitions. Also add a parallel statement and the ONLINE keyword:
SELECT 'alter index ' || IDX.TABLE_OWNER || '.' ||  IDX.INDEX_NAME || ' rebuild subpartition ' || IDXSUBPART.SUBPARTITION_NAME || ' parallel 4 tablespace ' || IDXSUBPART.TABLESPACE_NAME ||' online;'
FROM    DBA_INDEXES IDX, DBA_IND_SUBPARTITIONS IDXSUBPART
WHERE   IDX.TABLE_OWNER IN ('SCOTT','JAMES')
AND     IDX.INDEX_NAME = IDXSUBPART.INDEX_NAME
AND     IDX.INDEX_NAME IN (SELECT UNIQUE INDEX_NAME
                                        FROM DBA_IND_SUBPARTITIONS
                                        WHERE STATUS = 'UNUSABLE')
AND INDEX_OWNER NOT IN ('SYS','SYSTEM')
AND IDXSUBPART.STATUS <> 'USABLE'
ORDER BY IDX.OWNER, IDX.INDEX_NAME
/
Upon completion, the file "rebuild_stmts.sql" should contain your "ALTER INDEX REBUILD PARTITION..." directives.