Showing posts with label Postgres. Show all posts
Showing posts with label Postgres. Show all posts

Monday, November 10, 2025

Generate truncate table statements in PostgreSQL

To generate a script in postgreSQL, equivalent to the Oracle-style shown below:
select 'truncate table ' || table_name || ' cascade;' from dba_tables where owner='MYSCHEMA';
, put this in a file called gen_truncate.sql
/*
| Setting                  | Effect                                                    |
| ------------------------ | --------------------------------------------------------- |
| `\o /path/to/file`       | Redirects all query output to the file                    |
| `\pset format unaligned` | Produces plain text output (no table formatting)          |
| `\pset tuples_only on`   | Suppresses headers and row counts                         |
| `\pset footer off`       | Removes `x rows` footer                                   |
| `\pset border 0`         | Removes any border formatting (mostly for aligned format) |
*/

\o truncate_tables.sql
\pset format unaligned
\pset tuples_only on
\pset footer off
\pset border 0
SELECT 'TRUNCATE TABLE ' || schemaname || '.' || tablename || ' CASCADE;' AS stmt
FROM pg_tables
WHERE schemaname = 'myschema';
\o
Execute it:
psql mydb -f gen_truncate.sql
or login to the desired database directly as the owner of the table:
psql -h localhost -d mydb -U myuser -f gen_truncate.sql

Do I need special privileges to create temporary tables in a database in PostgreSQL?

To verify if your user has the privilege to create temporary segments in a postgreSQL database, use this check:
psql -h localhost -d mydb -U myuser

 SELECT has_database_privilege(current_user, current_database(), 'TEMP');
If the value returned shows:
has_database_privilege
------------------------
 f
To be able to create a temporary table in this database, my user needs privileges to do so:
psql
postgres=# grant temp on database mydb to myser;
GRANT
The same test will now yield true instead of false:
SELECT has_database_privilege(current_user, current_database(), 'TEMP');
 has_database_privilege
------------------------
 t
(1 row)
I can now create a temporary table:
CREATE TEMP TABLE table_counts (table_name text, row_count bigint);
Before the grant, the access privileges for my database was
mydb=> \l+ mydb

  |      Access privileges       |
  +------------------------------+
  | postgres=CTc/postgres       +|
  | myuser=c/postgres           +|
 
After the grant:
  |      Access privileges       |
  +------------------------------+
  | postgres=CTc/postgres       +|
  | myuser=Tc/postgres          +|
Notice how the privileges for "myuser" has a "T" ammended to it. This indicate the permission to create temporary objects.

Friday, October 31, 2025

PostgreSQL query that mimics Oracle's DBA_OBJECTS aggregation

with base_objects as (
  select 
    case 
      when c.relkind = 'r' then 'TABLE'
      when c.relkind = 'v' then 'VIEW'
      when c.relkind = 'm' then 'MATERIALIZED VIEW'
      when c.relkind = 'i' then 'INDEX'
      when c.relkind = 'S' then 'SEQUENCE'
      when c.relkind = 'f' then 'FOREIGN TABLE'
      when c.relkind = 'p' then 'PARTITIONED TABLE'
      when c.relkind = 'I' then 'PARTITIONED INDEX'
      else 'OTHER'
    end as object_type
  from pg_class c
  join pg_namespace n on n.oid = c.relnamespace
  where n.nspname = 'myschema'

  union all

  select 
    case p.prokind
      when 'f' then 'FUNCTION'
      when 'p' then 'PROCEDURE'
      when 'a' then 'AGGREGATE'
      when 'w' then 'WINDOW'
      else 'OTHER'
    end as object_type
  from pg_proc p
  join pg_namespace n on n.oid = p.pronamespace
  where n.nspname = 'myschema'

  union all
  select 'TYPE'
  from pg_type t
  join pg_namespace n on n.oid = t.typnamespace
  where n.nspname = 'myschema'
    and t.typtype in ('c', 'e', 'd') -- composite, enum, domain

  union all

  select 'CONSTRAINT'
  from pg_constraint c
  join pg_namespace n on n.oid = c.connamespace
  where n.nspname = 'myschema'
)

select object_type, count(*)
from base_objects
group by object_type
order by object_type;

Monday, September 29, 2025

How to pass parameters to a psql script

To use variables in your sql script, use the -v option to set variables on the command line.

Consider this example:
export PGPASSWORD=mysecretpassword
psql -h myserver.oric.no -d db01 -U scott -v schema=scott -v table=mytable -f myquery.sql
The query in myquery.sql can then referance the "schema" and "table" variables, like this:
SELECT
    att.attname AS column_name
FROM
    pg_attribute att
JOIN
    pg_class cls ON cls.oid = att.attrelid
JOIN
    pg_namespace ns ON ns.oid = cls.relnamespace
WHERE
    ns.nspname = :'schema'
    AND cls.relname = :'table'
    AND att.attnotnull
    AND att.attnum > 0
    AND NOT att.attisdropped;

Thursday, September 25, 2025

Find out if a constraint in PostgreSQL is defined as deferrable

Logged into the relevant database, find out which FK are deferrable:
SELECT conname, condeferrable, condeferred
FROM pg_constraint
WHERE conrelid in ( 'mytable1'::regclass,'mytable2'::regclass) 
AND contype='f';
           conname            | condeferrable | condeferred
------------------------------+---------------+-------------
 mytable1_id_fkey             | f             | f
 mytable1_id_fkey             | f             | f
 mytable2_id_fkey             | f             | f
  • If condeferrable = t (true), the constraint is created initially deferrable
  • If condeferrable = f (false), like in my case, it is not.
  • Wednesday, September 24, 2025

    Terminate PostgresSQL sessions

    -- Terminate all connections from user "sales"
    SELECT pg_terminate_backend(pid)
    FROM pg_stat_activity
    WHERE usename = 'sales';
    
    In my case the result was:
     pg_terminate_backend
    ----------------------
     t
     t
    (2 rows)
    

    Check active sessions in PostgreSQL

    The closest equivalent to this oracle query:
    SELECT USERNAME, COUNT(*) 
    FROM v$session 
    WHERE type <> 'BACKGROUND' 
    GROUP BY username;
    
    Would be
    SELECT datname, usename AS username, COUNT(*)
    FROM pg_stat_activity
    WHERE backend_type = 'client backend'
    GROUP BY datname, usename
    ORDER BY datname, usename;
    
    Example output:
     datname  |     username      | count
    ----------+-------------------+-------
     postgres | postgres          |     1
     postgres | postgres_exporter |     1
     mydb     | myser             |     2
    (3 rows)
    

    Tuesday, September 9, 2025

    How to set echo in a SQL script executed in psql

    Use one of two options 1. in the sql file, add the following directive
    \set ECHO queries
    
    2. execute the script using the -e flag
    psql -e -f myfile.sql
    

    Monday, September 8, 2025

    Syntax for drop/recreate of constraints in PostgreSQL

    1. Find the constraints belonging to a specific table:
    SELECT conname
    FROM pg_constraint
    WHERE conrelid = 'mytable'::regclass;
        conname
    ----------------
     sys_c002733781
     sys_c002733782
    (2 rows)
    
    2. Drop them:
    ALTER TABLE mytable DROP CONSTRAINT sys_c002733781;
    ALTER TABLE
    ALTER TABLE mytable DROP CONSTRAINT sys_c002733782;
    ALTER TABLE
    
    3. Load your data using your preferred method

    4. Recreate the FK constraints but skip validating existing data (only new inserts/updates are checked):
    alter table mytable add constraint sys_c002733781 foreign key(job_id) references jobs(id) on delete cascade not valid;
    ALTER TABLE
    
    alter table mytable add constraint sys_c002733782 foreign key(task_id) references tasks(id) on delete cascade not valid;
    ALTER TABLE
    
    5. Validate constraints:
    alter table mytable validate constraint sys_c002733781;
    ALTER TABLE
    alter table mytable validate constraint sys_c002733782;
    ALTER TABLE
    

    Friday, September 5, 2025

    Query equivalent to SELECT * FROM USER_OBJECTS in PostgreSQL

    SELECT schema_name, object_name, object_type
    FROM (
        -- Tables, Views, Indexes, etc.
        SELECT
            n.nspname AS schema_name,
            c.relname AS object_name,
            CASE c.relkind
                WHEN 'r' THEN 'TABLE'
                WHEN 'v' THEN 'VIEW'
                WHEN 'm' THEN 'MATERIALIZED VIEW'
                WHEN 'i' THEN 'INDEX'
                WHEN 'S' THEN 'SEQUENCE'
                WHEN 'f' THEN 'FOREIGN TABLE'
                WHEN 'p' THEN 'PARTITIONED TABLE'
                WHEN 'I' THEN 'PARTITIONED INDEX'
                ELSE c.relkind::text
            END AS object_type
        FROM pg_class c
        JOIN pg_namespace n ON n.oid = c.relnamespace
        WHERE n.nspname = current_schema()
    
        UNION ALL
    
        -- Functions
        SELECT
            n.nspname AS schema_name,
            p.proname AS object_name,
            'FUNCTION' AS object_type
        FROM pg_proc p
        JOIN pg_namespace n ON n.oid = p.pronamespace
        WHERE n.nspname = current_schema()
    ) AS objects
    ORDER BY object_type, object_name;
    
    It only shows objects in the current schema (like Oracle USER_OBJECTS).

    If you want all objects the user owns (across schemas), replace
    WHERE n.nspname = current_schema()
    
    with
    WHERE n.nspname NOT IN ('pg_catalog', 'information_schema')
    

    Thursday, August 28, 2025

    Spool out all parameters from a datafile to a file

    psql -h postgres01.oric.no -U mysuperuser postgres -c "COPY (SELECT name || '='  || setting FROM pg_settings ORDER BY name) TO STDOUT WITH CSV HEADER"   > pg_config.csv
    

    Wednesday, August 27, 2025

    List a table, its partitions and the number of rows it contains

    SELECT
        s.schemaname,
        s.relname AS table_name,
        s.n_live_tup,
        s.last_analyze,
        s.last_autoanalyze
    FROM
        pg_stat_all_tables s
    JOIN
        pg_class c ON c.relname = s.relname
    LEFT JOIN
        pg_inherits i ON i.inhrelid = c.oid OR i.inhparent = c.oid
    WHERE
        s.schemaname = 'myschema'
        AND (
            s.relname = 'mytable'
            OR c.oid IN (
                SELECT inhrelid
                FROM pg_inherits
                WHERE inhparent = (
                    SELECT oid FROM pg_class WHERE relname = 'mytable'
                )
            )
        )
    ORDER BY
     s.n_live_tup DESC,s.last_analyze;
    
    
    Example output
    
     schemaname      |  table_name     | n_live_tup | last_analyze   		| last_autoanalyze 
    -----------------+-----------------+------------+-------------------------------+------------------------------
     myschema 	 | mytable_2015_02 |   95788115 | 2025-08-06 12:58:22.952592+00 | 
     myschema 	 | mytable_2015_03 |   78505350 | 2025-08-06 12:58:37.147433+00 | 
     myschema 	 | mytable_2015_04 |   71211253 | 2025-08-06 12:58:51.311452+00 | 
     myschema 	 | mytable_2015_01 |   68255510 | 2025-08-06 12:58:08.42708+00  | 
     myschema 	 | mytable_2015_05 |   62075043 | 2025-08-06 12:59:05.434118+00 | 
     myschema 	 | mytable_2015_06 |   60888876 | 2025-08-06 12:59:19.918657+00 | 2025-08-06 11:31:19.551345+00
     myschema 	 | mytable_2015_12 |   57485594 | 2025-08-06 13:00:43.112316+00 | 2025-08-27 03:13:11.394606+00
     myschema 	 | mytable_2016_01 |   57405307 | 2025-08-06 13:00:55.395113+00 | 2025-08-25 13:35:07.749773+00
    

    Thursday, July 31, 2025

    How to check if a PostgreSQL streaming replicate server is up-to-date with the primary

    On the primary, run
    SELECT pg_current_wal_lsn(); -- On the primary
    
    In my case, the result was
     pg_current_wal_lsn
    --------------------
     0/E015420
    
    On the standby, run
    SELECT pg_last_wal_replay_lsn();
    
    In my case, the result was
    pg_last_wal_replay_lsn
    ------------------------
     0/E015420
    

    This means that

  • All WAL records have been received
  • All WAL recores have been replayed (not just flushed) on the standby
  • There is no lag between primary and standby
  • Basic postgres connect syntax

    psql -h hostname -U username -d database
    
    Options:
    Option	Description
    -h <host>	Hostname or IP address of the PostgreSQL server
    -p <port>	Port number (default is 5432)
    -U <username>	Username to connect as
    -d <dbname>	Database name
    -W Prompt for password (useful if password auth is required)
    -f <filename>	Execute commands from a SQL script file
    -c "<command>"	Execute a single SQL command and exit
    -v var=value	Set psql variable (used inside scripts)
    --set=var=value	Same as -v, more explicit
    --dbname=<connstr>	Full connection string, e.g.: postgresql://user:pass@host:port/dbname
    
    Example
    psql -h pg01.oric.no -p 5432 -U myuser -d mydb
    
    Alternatively, use an URL, which is more scripting friendly:
    psql "postgresql://replicate@pg01.oric.no:5432/mydb"
    

    Wednesday, July 30, 2025

    How to find the columns in a specific postgres data dictionary table

    For example, to check which columns belongs to the view "pg_stat_wal_receiver"
    SELECT column_name
    FROM information_schema.columns
    WHERE table_name = 'pg_stat_wal_receiver';
    
    Output
         column_name
    -----------------------
     pid
     last_msg_receipt_time
     latest_end_lsn
     latest_end_time
     sender_port
     receive_start_lsn
     receive_start_tli
     written_lsn
     flushed_lsn
     received_tli
     last_msg_send_time
     status
     slot_name
     sender_host
     conninfo
    (15 rows)
    
    Or use the psql metacommand "\d+":
    echo '\d+ pg_stat_wal_receiver' |psql
    
    Output
                                      View "pg_catalog.pg_stat_wal_receiver"
            Column         |           Type           | Collation | Nullable | Default | Storage  | Description
    -----------------------+--------------------------+-----------+----------+---------+----------+-------------
     pid                   | integer                  |           |          |         | plain    |
     status                | text                     |           |          |         | extended |
     receive_start_lsn     | pg_lsn                   |           |          |         | plain    |
     receive_start_tli     | integer                  |           |          |         | plain    |
     written_lsn           | pg_lsn                   |           |          |         | plain    |
     flushed_lsn           | pg_lsn                   |           |          |         | plain    |
     received_tli          | integer                  |           |          |         | plain    |
     last_msg_send_time    | timestamp with time zone |           |          |         | plain    |
     last_msg_receipt_time | timestamp with time zone |           |          |         | plain    |
     latest_end_lsn        | pg_lsn                   |           |          |         | plain    |
     latest_end_time       | timestamp with time zone |           |          |         | plain    |
     slot_name             | text                     |           |          |         | extended |
     sender_host           | text                     |           |          |         | extended |
     sender_port           | integer                  |           |          |         | plain    |
     conninfo              | text                     |           |          |         | extended |
    View definition:
     SELECT pid,
        status,
        receive_start_lsn,
        receive_start_tli,
        written_lsn,
        flushed_lsn,
        received_tli,
        last_msg_send_time,
        last_msg_receipt_time,
        latest_end_lsn,
        latest_end_time,
        slot_name,
        sender_host,
        sender_port,
        conninfo
       FROM pg_stat_get_wal_receiver() s(pid, status, receive_start_lsn, receive_start_tli, written_lsn, flushed_lsn, received_tli, last_msg_send_time, last_msg_receipt_time, latest_end_lsn, latest_end_time, slot_name, sender_host, sender_port, conninfo)
      WHERE pid IS NOT NULL;
    

    Monday, July 28, 2025

    How to list the postgres processes running on your server and what their jobs are

    A neet little trick I learned from the series "Mirage of memory, part 2: PSS"
    ps -fu postgres | grep $(pgrep -f postmaster)
    postgres    1913       1  0 Jul17 ?        00:02:51 /usr/pgsql-15/bin/postmaster -D /var/lib/pgsql/15/data/
    postgres    1982    1913  0 Jul17 ?        00:00:24 postgres: logger
    postgres    2063    1913  0 Jul17 ?        00:00:18 postgres: checkpointer
    postgres    2064    1913  0 Jul17 ?        00:00:03 postgres: background writer
    postgres    2119    1913  0 Jul17 ?        00:00:07 postgres: walwriter
    postgres    2120    1913  0 Jul17 ?        00:00:14 postgres: autovacuum launcher
    postgres    2121    1913  0 Jul17 ?        00:00:01 postgres: archiver
    postgres    2122    1913  0 Jul17 ?        00:00:00 postgres: logical replication launcher
    postgres    2657    1913  0 Jul17 ?        00:02:05 postgres: postgres_exporter postgres localhost(48674) idle
    
    The command lists all processes by user "postgres" and and greps for the output of the command "pgrep -f postmaster", which returns a list of process IDs

    You could also count the number of processes by user "postgres" by simply typing
    pgrep -c postmaster
    9
    

    Monday, July 14, 2025

    How to find when a PostgreSQL table was last vacuumed

    SELECT
    schemaname,
    relname,
    last_vacuum,
    last_autovacuum,
    last_analyze,
    last_autoanalyze
    FROM pg_stat_user_tables
    WHERE relname = 'documents';
       schemaname   | relname  | last_vacuum |        last_autovacuum        |         last_analyze          |       last_autoanalyze
    ----------------+----------+-------------+-------------------------------+-------------------------------+-------------------------------
     scott          | documents|             | 2025-06-25 02:05:20.692869+02 | 2025-07-12 22:04:05.119051+02 | 2025-06-17 08:41:12.567351+02
    (1 row)
    
    Explanation of columns:

  • last_vacuum: Last manual vacuum.
  • last_autovacuum: Last automatic vacuum.
  • last_analyze: Last manual analyze.
  • last_autoanalyze: Last automatic analyze.

    What does ANALYZE do in PostgreSQL?

    ANALYZE collects statistics about the contents of a table (or all tables), such as:
  • Number of rows
  • Distribution of values
  • Most common values
  • Histogram of value ranges
  • Null fractions These statistics are stored in pg_statistic and are used by the query planner to make decisions (e.g., which index to use, join strategies, etc.).

    Syntax to run ANALYZE

    Analyze the entire database (all tables in all schemas):
    ANALYZE;
    
    Analyze a specific table:
    ANALYZE my_table;
    
    Analyze specific columns only:
    ANALYZE my_table (column1, column2);
    
  • Friday, July 11, 2025

    PostgreSQL: how to tail the alert log of the day with filtering for long-running statements

    I have set the parameter log_min_duration_statement to 2 seconds and reloaded the config:
    postgres=# alter system set log_min_duration_statement='2s';
    ALTER SYSTEM
    
    postgres=# SELECT pg_reload_conf();
     pg_reload_conf
    ----------------
     t
    (1 row)
    
    postgres=# show log_min_duration_statement;
     log_min_duration_statement
    ----------------------------
     2s
    (1 row)
    
    
    I expect every statement that lasts longer than 2 seconds to be logged in the postgreSQL server main log.

    From a different session, I execute the following in any database in the cluster:
    postgres=# SELECT pg_sleep(3);
     pg_sleep
    ----------
    
    (1 row)
    
    From another session, tail the log from today:
    tail -F /var/log/postgres/postgresql-2025-07-11_000000.log | grep --line-buffered "duration:.*statement:"
    
    Output
    2025-07-11 09:45:42 CEST [173349]: [5-1] user=postgres,db=postgres,app=psql,client=[local] LOG:  duration: 3003.698 ms  statement: SELECT pg_sleep(3);
    

    Thursday, June 19, 2025

    Postgres: List all RANGE partitions, sorted by their range start value

    My table contains 180 partitons, and looks like this when described in psql:
    mydb=> \d mytable
          Partitioned table "myschema.mytable"
          Column       |           Type           | Collation | Nullable |           Default            
    -------------------+--------------------------+-----------+----------+------------------------------
     id                | bigint                   |           | not null | generated always as identity
     ad_updtud_log_id  | bigint                   |           | not null | 
     period            | integer                  |           | not null | 
     created           | timestamp with time zone |           | not null | 
    Partition key: RANGE (id)
    Indexes:
        "mytable_pkey" PRIMARY KEY, btree (id, periode)
        "mytable_id_index" btree (im_oppsum_logg_id)
        "mytable_opprettet_index" btree (opprettet)
    Foreign-key constraints:
        "fk_mytable" FOREIGN KEY (ad_updtud_log_id, period) REFERENCES mytable2(id, period)
    Number of partitions: 180 (Use \d+ to list them.)
    
    This is how you can list all partitions belonging to a range-partitioned table in postgreSQL, based on the values:
    SELECT
        child.relname AS partition_name,
        regexp_replace(pg_get_expr(child.relpartbound, child.oid), '.*FROM \(''(\d+).*', '\1')::bigint AS range_start,
        pg_get_expr(child.relpartbound, child.oid) AS partition_range
    FROM pg_inherits
    JOIN pg_class parent ON pg_inherits.inhparent = parent.oid
    JOIN pg_class child ON pg_inherits.inhrelid = child.oid
    JOIN pg_namespace nm ON nm.oid = child.relnamespace
    WHERE parent.relname = 'mytable'
      AND nm.nspname = 'myschema'
    ORDER BY range_start;
    
    Result (excerpt only):
       partition_name   | range_start | partition_range                  
    ---------------------------------------+-------------+--------------------------------------------
     mytable_0	    |           0 | FOR VALUES FROM ('0') TO ('30000000')
     mytable_30000000   |    30000000 | FOR VALUES FROM ('30000000') TO ('60000000')
     mytable_60000000   |    60000000 | FOR VALUES FROM ('60000000') TO ('90000000')
     mytable_90000000   |    90000000 | FOR VALUES FROM ('90000000') TO ('120000000')
     mytable_120000000  |   120000000 | FOR VALUES FROM ('120000000') TO ('150000000')
     mytable_150000000  |   150000000 | FOR VALUES FROM ('150000000') TO ('180000000')
     mytable_180000000  |   180000000 | FOR VALUES FROM ('180000000') TO ('210000000')
     mytable_210000000  |   210000000 | FOR VALUES FROM ('210000000') TO ('240000000')
     mytable_240000000  |   240000000 | FOR VALUES FROM ('240000000') TO ('270000000')
     mytable_270000000  |   270000000 | FOR VALUES FROM ('270000000') TO ('300000000')
     mytable_300000000  |   300000000 | FOR VALUES FROM ('300000000') TO ('330000000')
     mytable_330000000  |   330000000 | FOR VALUES FROM ('330000000') TO ('360000000')
     mytable_360000000  |   360000000 | FOR VALUES FROM ('360000000') TO ('390000000')
     mytable_390000000  |   390000000 | FOR VALUES FROM ('390000000') TO ('420000000')
     mytable_420000000  |   420000000 | FOR VALUES FROM ('420000000') TO ('450000000')
     mytable_450000000  |   450000000 | FOR VALUES FROM ('450000000') TO ('480000000')
     

    Thursday, June 5, 2025

    PostgreSQL search_path basics

    Similar to Oracles alter session set current_schema=<schema_name>, in PostgreSQL, search_path is a session-level setting that determines the order in which schemas are searched when you reference database objects without a schema name. It consists of a list of schema names. When you run a query like
    SELECT * FROM mytable;
    PostgreSQL checks each schema in the list — in order — to find mytable.

    Default value is:
     "$user", public 
    This means:
    1. Look for a schema named after the current user.
    2. If not found or not accessible, try the public schema.

    How to view and set it

    Check your current search_path:
     SHOW search_path; 
    Set it for the session:
     SET search_path TO schema1, public; 
    or
     SET search_path TO schema1, schema2, public; 
    The last example is important in cases where you logon to a database with a user with no matching schema. Consider the following example:
     psql -h server1.oric.no -U jim -d musicdb
    
    I am logging onto the database "musicdb" with a user called "jim". By default, jim will have its own user followed by public, in his search path:
    musicdb=> show search_path;
       search_path
    -----------------
     "$user", public
    (1 row)
    
    I have already given user jim the privilges needed to "see" the objects created by schema "music" which exists in the database "musicdb".

    For convenience, add schema "music" to the search_path:
    musicdb=> set search_path to 'music','$user','public';
    SET
    musicdb=> show search_path;
              search_path
    -------------------------------
     music, "$user", public
    (1 row)
    
    The "current_user" and "current_schema" functions will now return the actual user name, and the first match in the search_path, respectivly:
    
    musicdb=> select current_user, current_schema;
     current_user | current_schema
    --------------+----------------
     jim          | music
    (1 row)
    
    Why is it important?
    It controls where PostgreSQL looks first for unqualified object names. It allows you to skip schema prefixes when working with other schemas' objects