Showing posts with label Postgres. Show all posts
Showing posts with label Postgres. Show all posts

Thursday, November 20, 2025

Find total size of all databases in PostgreSQL cluster

SELECT database, size FROM (
  SELECT datname AS database,
         pg_size_pretty(pg_database_size(datname)) AS size,
         0 AS sort_order
  FROM pg_database
  UNION ALL
  SELECT 'TOTAL',
         pg_size_pretty(SUM(pg_database_size(datname))),
         1
  FROM pg_database
) AS sub
ORDER BY sort_order, size DESC;
Example output:
             database             |  size
----------------------------------+---------
 mydb01                           | 7819 kB
 mydb02                           | 7795 kB
 postgres                         | 7739 kB
 template0                        | 7731 kB
 template1                        | 7715 kB
 proddb01                         | 76 GB
 proddb02                         | 2971 GB
 proddb03                         | 22 GB
 warehouse01                      | 11 TB
 testdb01                         | 106 MB
 TOTAL                            | 14 TB
(11 rows)

Monday, November 17, 2025

Monday, November 10, 2025

Generate truncate table statements in PostgreSQL

To generate a script in postgreSQL, equivalent to the Oracle-style shown below:
select 'truncate table ' || table_name || ' cascade;' from dba_tables where owner='MYSCHEMA';
, put this in a file called gen_truncate.sql
/*
| Setting                  | Effect                                                    |
| ------------------------ | --------------------------------------------------------- |
| `\o /path/to/file`       | Redirects all query output to the file                    |
| `\pset format unaligned` | Produces plain text output (no table formatting)          |
| `\pset tuples_only on`   | Suppresses headers and row counts                         |
| `\pset footer off`       | Removes `x rows` footer                                   |
| `\pset border 0`         | Removes any border formatting (mostly for aligned format) |
*/

\o truncate_tables.sql
\pset format unaligned
\pset tuples_only on
\pset footer off
\pset border 0
SELECT 'TRUNCATE TABLE ' || schemaname || '.' || tablename || ' CASCADE;' AS stmt
FROM pg_tables
WHERE schemaname = 'myschema';
\o
Execute it:
psql mydb -f gen_truncate.sql
or login to the desired database directly as the owner of the table:
psql -h localhost -d mydb -U myuser -f gen_truncate.sql

Do I need special privileges to create temporary tables in a database in PostgreSQL?

To verify if your user has the privilege to create temporary segments in a postgreSQL database, use this check:
psql -h localhost -d mydb -U myuser

 SELECT has_database_privilege(current_user, current_database(), 'TEMP');
If the value returned shows:
has_database_privilege
------------------------
 f
To be able to create a temporary table in this database, my user needs privileges to do so:
psql
postgres=# grant temp on database mydb to myser;
GRANT
The same test will now yield true instead of false:
SELECT has_database_privilege(current_user, current_database(), 'TEMP');
 has_database_privilege
------------------------
 t
(1 row)
I can now create a temporary table:
CREATE TEMP TABLE table_counts (table_name text, row_count bigint);
Before the grant, the access privileges for my database was
mydb=> \l+ mydb

  |      Access privileges       |
  +------------------------------+
  | postgres=CTc/postgres       +|
  | myuser=c/postgres           +|
 
After the grant:
  |      Access privileges       |
  +------------------------------+
  | postgres=CTc/postgres       +|
  | myuser=Tc/postgres          +|
Notice how the privileges for "myuser" has a "T" ammended to it. This indicate the permission to create temporary objects.

Friday, October 31, 2025

PostgreSQL query that mimics Oracle's DBA_OBJECTS aggregation

with base_objects as (
  select 
    case 
      when c.relkind = 'r' then 'TABLE'
      when c.relkind = 'v' then 'VIEW'
      when c.relkind = 'm' then 'MATERIALIZED VIEW'
      when c.relkind = 'i' then 'INDEX'
      when c.relkind = 'S' then 'SEQUENCE'
      when c.relkind = 'f' then 'FOREIGN TABLE'
      when c.relkind = 'p' then 'PARTITIONED TABLE'
      when c.relkind = 'I' then 'PARTITIONED INDEX'
      else 'OTHER'
    end as object_type
  from pg_class c
  join pg_namespace n on n.oid = c.relnamespace
  where n.nspname = 'myschema'

  union all

  select 
    case p.prokind
      when 'f' then 'FUNCTION'
      when 'p' then 'PROCEDURE'
      when 'a' then 'AGGREGATE'
      when 'w' then 'WINDOW'
      else 'OTHER'
    end as object_type
  from pg_proc p
  join pg_namespace n on n.oid = p.pronamespace
  where n.nspname = 'myschema'

  union all
  select 'TYPE'
  from pg_type t
  join pg_namespace n on n.oid = t.typnamespace
  where n.nspname = 'myschema'
    and t.typtype in ('c', 'e', 'd') -- composite, enum, domain

  union all

  select 'CONSTRAINT'
  from pg_constraint c
  join pg_namespace n on n.oid = c.connamespace
  where n.nspname = 'myschema'
)

select object_type, count(*)
from base_objects
group by object_type
order by object_type;

Monday, September 29, 2025

How to pass parameters to a psql script

To use variables in your sql script, use the -v option to set variables on the command line.

Consider this example:
export PGPASSWORD=mysecretpassword
psql -h myserver.oric.no -d db01 -U scott -v schema=scott -v table=mytable -f myquery.sql
The query in myquery.sql can then referance the "schema" and "table" variables, like this:
SELECT
    att.attname AS column_name
FROM
    pg_attribute att
JOIN
    pg_class cls ON cls.oid = att.attrelid
JOIN
    pg_namespace ns ON ns.oid = cls.relnamespace
WHERE
    ns.nspname = :'schema'
    AND cls.relname = :'table'
    AND att.attnotnull
    AND att.attnum > 0
    AND NOT att.attisdropped;

Thursday, September 25, 2025

Find out if a constraint in PostgreSQL is defined as deferrable

Logged into the relevant database, find out which FK are deferrable:
SELECT conname, condeferrable, condeferred
FROM pg_constraint
WHERE conrelid in ( 'mytable1'::regclass,'mytable2'::regclass) 
AND contype='f';
           conname            | condeferrable | condeferred
------------------------------+---------------+-------------
 mytable1_id_fkey             | f             | f
 mytable1_id_fkey             | f             | f
 mytable2_id_fkey             | f             | f
  • If condeferrable = t (true), the constraint is created initially deferrable
  • If condeferrable = f (false), like in my case, it is not.
  • Wednesday, September 24, 2025

    Terminate PostgresSQL sessions

    -- Terminate all connections from user "sales"
    SELECT pg_terminate_backend(pid)
    FROM pg_stat_activity
    WHERE usename = 'sales';
    
    In my case the result was:
     pg_terminate_backend
    ----------------------
     t
     t
    (2 rows)
    

    Check active sessions in PostgreSQL

    The closest equivalent to this oracle query:
    SELECT USERNAME, COUNT(*) 
    FROM v$session 
    WHERE type <> 'BACKGROUND' 
    GROUP BY username;
    
    Would be
    SELECT datname, usename AS username, COUNT(*)
    FROM pg_stat_activity
    WHERE backend_type = 'client backend'
    GROUP BY datname, usename
    ORDER BY datname, usename;
    
    Example output:
     datname  |     username      | count
    ----------+-------------------+-------
     postgres | postgres          |     1
     postgres | postgres_exporter |     1
     mydb     | myser             |     2
    (3 rows)
    

    Tuesday, September 9, 2025

    How to set echo in a SQL script executed in psql

    Use one of two options 1. in the sql file, add the following directive
    \set ECHO queries
    
    2. execute the script using the -e flag
    psql -e -f myfile.sql
    

    Monday, September 8, 2025

    Syntax for drop/recreate of constraints in PostgreSQL

    1. Find the constraints belonging to a specific table:
    SELECT conname
    FROM pg_constraint
    WHERE conrelid = 'mytable'::regclass;
        conname
    ----------------
     sys_c002733781
     sys_c002733782
    (2 rows)
    
    2. Drop them:
    ALTER TABLE mytable DROP CONSTRAINT sys_c002733781;
    ALTER TABLE
    ALTER TABLE mytable DROP CONSTRAINT sys_c002733782;
    ALTER TABLE
    
    3. Load your data using your preferred method

    4. Recreate the FK constraints but skip validating existing data (only new inserts/updates are checked):
    alter table mytable add constraint sys_c002733781 foreign key(job_id) references jobs(id) on delete cascade not valid;
    ALTER TABLE
    
    alter table mytable add constraint sys_c002733782 foreign key(task_id) references tasks(id) on delete cascade not valid;
    ALTER TABLE
    
    5. Validate constraints:
    alter table mytable validate constraint sys_c002733781;
    ALTER TABLE
    alter table mytable validate constraint sys_c002733782;
    ALTER TABLE
    

    Friday, September 5, 2025

    Query equivalent to SELECT * FROM USER_OBJECTS in PostgreSQL

    SELECT schema_name, object_name, object_type
    FROM (
        -- Tables, Views, Indexes, etc.
        SELECT
            n.nspname AS schema_name,
            c.relname AS object_name,
            CASE c.relkind
                WHEN 'r' THEN 'TABLE'
                WHEN 'v' THEN 'VIEW'
                WHEN 'm' THEN 'MATERIALIZED VIEW'
                WHEN 'i' THEN 'INDEX'
                WHEN 'S' THEN 'SEQUENCE'
                WHEN 'f' THEN 'FOREIGN TABLE'
                WHEN 'p' THEN 'PARTITIONED TABLE'
                WHEN 'I' THEN 'PARTITIONED INDEX'
                ELSE c.relkind::text
            END AS object_type
        FROM pg_class c
        JOIN pg_namespace n ON n.oid = c.relnamespace
        WHERE n.nspname = current_schema()
    
        UNION ALL
    
        -- Functions
        SELECT
            n.nspname AS schema_name,
            p.proname AS object_name,
            'FUNCTION' AS object_type
        FROM pg_proc p
        JOIN pg_namespace n ON n.oid = p.pronamespace
        WHERE n.nspname = current_schema()
    ) AS objects
    ORDER BY object_type, object_name;
    
    It only shows objects in the current schema (like Oracle USER_OBJECTS).

    If you want all objects the user owns (across schemas), replace
    WHERE n.nspname = current_schema()
    
    with
    WHERE n.nspname NOT IN ('pg_catalog', 'information_schema')
    

    Thursday, August 28, 2025

    Spool out all parameters from a datafile to a file

    psql -h postgres01.oric.no -U mysuperuser postgres -c "COPY (SELECT name || '='  || setting FROM pg_settings ORDER BY name) TO STDOUT WITH CSV HEADER"   > pg_config.csv
    

    Wednesday, August 27, 2025

    List a table, its partitions and the number of rows it contains

    SELECT
        s.schemaname,
        s.relname AS table_name,
        s.n_live_tup,
        s.last_analyze,
        s.last_autoanalyze
    FROM
        pg_stat_all_tables s
    JOIN
        pg_class c ON c.relname = s.relname
    LEFT JOIN
        pg_inherits i ON i.inhrelid = c.oid OR i.inhparent = c.oid
    WHERE
        s.schemaname = 'myschema'
        AND (
            s.relname = 'mytable'
            OR c.oid IN (
                SELECT inhrelid
                FROM pg_inherits
                WHERE inhparent = (
                    SELECT oid FROM pg_class WHERE relname = 'mytable'
                )
            )
        )
    ORDER BY
     s.n_live_tup DESC,s.last_analyze;
    
    
    Example output
    
     schemaname      |  table_name     | n_live_tup | last_analyze   		| last_autoanalyze 
    -----------------+-----------------+------------+-------------------------------+------------------------------
     myschema 	 | mytable_2015_02 |   95788115 | 2025-08-06 12:58:22.952592+00 | 
     myschema 	 | mytable_2015_03 |   78505350 | 2025-08-06 12:58:37.147433+00 | 
     myschema 	 | mytable_2015_04 |   71211253 | 2025-08-06 12:58:51.311452+00 | 
     myschema 	 | mytable_2015_01 |   68255510 | 2025-08-06 12:58:08.42708+00  | 
     myschema 	 | mytable_2015_05 |   62075043 | 2025-08-06 12:59:05.434118+00 | 
     myschema 	 | mytable_2015_06 |   60888876 | 2025-08-06 12:59:19.918657+00 | 2025-08-06 11:31:19.551345+00
     myschema 	 | mytable_2015_12 |   57485594 | 2025-08-06 13:00:43.112316+00 | 2025-08-27 03:13:11.394606+00
     myschema 	 | mytable_2016_01 |   57405307 | 2025-08-06 13:00:55.395113+00 | 2025-08-25 13:35:07.749773+00
    

    Thursday, July 31, 2025

    How to check if a PostgreSQL streaming replicate server is up-to-date with the primary

    On the primary, run
    SELECT pg_current_wal_lsn(); -- On the primary
    
    In my case, the result was
     pg_current_wal_lsn
    --------------------
     0/E015420
    
    On the standby, run
    SELECT pg_last_wal_replay_lsn();
    
    In my case, the result was
    pg_last_wal_replay_lsn
    ------------------------
     0/E015420
    

    This means that

  • All WAL records have been received
  • All WAL recores have been replayed (not just flushed) on the standby
  • There is no lag between primary and standby
  • Basic postgres connect syntax

    psql -h hostname -U username -d database
    
    Options:
    Option	Description
    -h <host>	Hostname or IP address of the PostgreSQL server
    -p <port>	Port number (default is 5432)
    -U <username>	Username to connect as
    -d <dbname>	Database name
    -W Prompt for password (useful if password auth is required)
    -f <filename>	Execute commands from a SQL script file
    -c "<command>"	Execute a single SQL command and exit
    -v var=value	Set psql variable (used inside scripts)
    --set=var=value	Same as -v, more explicit
    --dbname=<connstr>	Full connection string, e.g.: postgresql://user:pass@host:port/dbname
    
    Example
    psql -h pg01.oric.no -p 5432 -U myuser -d mydb
    
    Alternatively, use an URL, which is more scripting friendly:
    psql "postgresql://replicate@pg01.oric.no:5432/mydb"
    

    Wednesday, July 30, 2025

    How to find the columns in a specific postgres data dictionary table

    For example, to check which columns belongs to the view "pg_stat_wal_receiver"
    SELECT column_name
    FROM information_schema.columns
    WHERE table_name = 'pg_stat_wal_receiver';
    
    Output
         column_name
    -----------------------
     pid
     last_msg_receipt_time
     latest_end_lsn
     latest_end_time
     sender_port
     receive_start_lsn
     receive_start_tli
     written_lsn
     flushed_lsn
     received_tli
     last_msg_send_time
     status
     slot_name
     sender_host
     conninfo
    (15 rows)
    
    Or use the psql metacommand "\d+":
    echo '\d+ pg_stat_wal_receiver' |psql
    
    Output
                                      View "pg_catalog.pg_stat_wal_receiver"
            Column         |           Type           | Collation | Nullable | Default | Storage  | Description
    -----------------------+--------------------------+-----------+----------+---------+----------+-------------
     pid                   | integer                  |           |          |         | plain    |
     status                | text                     |           |          |         | extended |
     receive_start_lsn     | pg_lsn                   |           |          |         | plain    |
     receive_start_tli     | integer                  |           |          |         | plain    |
     written_lsn           | pg_lsn                   |           |          |         | plain    |
     flushed_lsn           | pg_lsn                   |           |          |         | plain    |
     received_tli          | integer                  |           |          |         | plain    |
     last_msg_send_time    | timestamp with time zone |           |          |         | plain    |
     last_msg_receipt_time | timestamp with time zone |           |          |         | plain    |
     latest_end_lsn        | pg_lsn                   |           |          |         | plain    |
     latest_end_time       | timestamp with time zone |           |          |         | plain    |
     slot_name             | text                     |           |          |         | extended |
     sender_host           | text                     |           |          |         | extended |
     sender_port           | integer                  |           |          |         | plain    |
     conninfo              | text                     |           |          |         | extended |
    View definition:
     SELECT pid,
        status,
        receive_start_lsn,
        receive_start_tli,
        written_lsn,
        flushed_lsn,
        received_tli,
        last_msg_send_time,
        last_msg_receipt_time,
        latest_end_lsn,
        latest_end_time,
        slot_name,
        sender_host,
        sender_port,
        conninfo
       FROM pg_stat_get_wal_receiver() s(pid, status, receive_start_lsn, receive_start_tli, written_lsn, flushed_lsn, received_tli, last_msg_send_time, last_msg_receipt_time, latest_end_lsn, latest_end_time, slot_name, sender_host, sender_port, conninfo)
      WHERE pid IS NOT NULL;
    

    Monday, July 28, 2025

    How to list the postgres processes running on your server and what their jobs are

    A neet little trick I learned from the series "Mirage of memory, part 2: PSS"
    ps -fu postgres | grep $(pgrep -f postmaster)
    postgres    1913       1  0 Jul17 ?        00:02:51 /usr/pgsql-15/bin/postmaster -D /var/lib/pgsql/15/data/
    postgres    1982    1913  0 Jul17 ?        00:00:24 postgres: logger
    postgres    2063    1913  0 Jul17 ?        00:00:18 postgres: checkpointer
    postgres    2064    1913  0 Jul17 ?        00:00:03 postgres: background writer
    postgres    2119    1913  0 Jul17 ?        00:00:07 postgres: walwriter
    postgres    2120    1913  0 Jul17 ?        00:00:14 postgres: autovacuum launcher
    postgres    2121    1913  0 Jul17 ?        00:00:01 postgres: archiver
    postgres    2122    1913  0 Jul17 ?        00:00:00 postgres: logical replication launcher
    postgres    2657    1913  0 Jul17 ?        00:02:05 postgres: postgres_exporter postgres localhost(48674) idle
    
    The command lists all processes by user "postgres" and and greps for the output of the command "pgrep -f postmaster", which returns a list of process IDs

    You could also count the number of processes by user "postgres" by simply typing
    pgrep -c postmaster
    9
    

    Monday, July 14, 2025

    How to find when a PostgreSQL table was last vacuumed

    SELECT
    schemaname,
    relname,
    last_vacuum,
    last_autovacuum,
    last_analyze,
    last_autoanalyze
    FROM pg_stat_user_tables
    WHERE relname = 'documents';
       schemaname   | relname  | last_vacuum |        last_autovacuum        |         last_analyze          |       last_autoanalyze
    ----------------+----------+-------------+-------------------------------+-------------------------------+-------------------------------
     scott          | documents|             | 2025-06-25 02:05:20.692869+02 | 2025-07-12 22:04:05.119051+02 | 2025-06-17 08:41:12.567351+02
    (1 row)
    
    Explanation of columns:

  • last_vacuum: Last manual vacuum.
  • last_autovacuum: Last automatic vacuum.
  • last_analyze: Last manual analyze.
  • last_autoanalyze: Last automatic analyze.

    What does ANALYZE do in PostgreSQL?

    ANALYZE collects statistics about the contents of a table (or all tables), such as:
  • Number of rows
  • Distribution of values
  • Most common values
  • Histogram of value ranges
  • Null fractions These statistics are stored in pg_statistic and are used by the query planner to make decisions (e.g., which index to use, join strategies, etc.).

    Syntax to run ANALYZE

    Analyze the entire database (all tables in all schemas):
    ANALYZE;
    
    Analyze a specific table:
    ANALYZE my_table;
    
    Analyze specific columns only:
    ANALYZE my_table (column1, column2);
    
  • Friday, July 11, 2025

    PostgreSQL: how to tail the alert log of the day with filtering for long-running statements

    I have set the parameter log_min_duration_statement to 2 seconds and reloaded the config:
    postgres=# alter system set log_min_duration_statement='2s';
    ALTER SYSTEM
    
    postgres=# SELECT pg_reload_conf();
     pg_reload_conf
    ----------------
     t
    (1 row)
    
    postgres=# show log_min_duration_statement;
     log_min_duration_statement
    ----------------------------
     2s
    (1 row)
    
    
    I expect every statement that lasts longer than 2 seconds to be logged in the postgreSQL server main log.

    From a different session, I execute the following in any database in the cluster:
    postgres=# SELECT pg_sleep(3);
     pg_sleep
    ----------
    
    (1 row)
    
    From another session, tail the log from today:
    tail -F /var/log/postgres/postgresql-2025-07-11_000000.log | grep --line-buffered "duration:.*statement:"
    
    Output
    2025-07-11 09:45:42 CEST [173349]: [5-1] user=postgres,db=postgres,app=psql,client=[local] LOG:  duration: 3003.698 ms  statement: SELECT pg_sleep(3);