Potential solution to ORA-1033 during configuration of standby database in data guard broker

In my environment, the following message was displayed in dgmgrl when trying to configure a data guard environment:

DGMGRL> add database 'hdal_stb' as  connect identifier is 'hdal_stb';
Error: ORA-1033: ORACLE initialization or shutdown in progress

Failed.

I had prior to this attempt duplicated the target database to the auxiliary using "duplicate target databas for standby from active database" - and the output looked fine.

Turns out that you cannot set the parameter redo_transport_user to any other value than SYS when cloning for standby.
In my environment, we use a dedicated user for this purpuse, as we avoid using the SYS user as much as possible.

I adjusted the parameter to read SYS on both the primary and standby instance, and reran the duplication.

Afterwards, the database could be added:

DGMGRL> add database 'hdal_stb' as  connect identifier is 'hdal_stb';
Database "hdal_stb" added

How to use mkstore to set up passwordless duplication

Oracle provides a method called "Secure External Password Store" to hide your passwords in a wallet, instead of using them directly in your scripts.

The feature Secure External Password Store can be used without any restriction in all product editions, you do not require a license for the Advanced Security Option (ASO).

/

What exactly is a "secure external password store"?
Oracle describes it as "a secure software container that stores authentication and signing credentials"
You need to use the utility mkstore to manage secure external password store.

Personally, I find them very useful when performing duplication. Here is how I set things up when I was cloning a database for standby.


NOTE: If you intend to clone for standby, you need to add the SYS user and nothing else to your wallet!
Any other privileged user will give you trouble during cloning.


On the standby server, create directory where you want the wallet to reside. I normall put it in the $TNS_ADMIN directory:

mkdir -p $TNS_ADMIN/wallet

Create the wallet:

mkstore -wrl $TNS_ADMIN/wallet -create
Oracle Secret Store Tool : Version 12.1.0.2
Copyright (c) 2004, 2014, Oracle and/or its affiliates. All rights reserved.

Enter password:
Enter password again:

Add a credential to one or more database

mkstore -wrl $TNS_ADMIN/wallet -createCredential hdal_primary dbaadmin
mkstore -wrl $TNS_ADMIN/wallet -createCredential hdal_stby dbaadmin

Note!
If you need to include a domain in your credential, use " " around your database name:

mkstore -wrl $TNS_ADMIN/wallet -createCredential "hdal_primary.mydomain.com" dbaadmin

If you later need to modify the same credential:

mkstore -wrl $TNS_ADMIN/wallet -modifyCredential hdal_stby dbaadmin

If you later need to delete a credential:

mkstore -wrl $TNS_ADMIN/wallet -deleteCredential hdal_stby

Verify that the credentials were added:

mkstore -wrl $TNS_ADMIN/wallet -listCredential
Oracle Secret Store Tool : Version 12.1.0.2
Copyright (c) 2004, 2014, Oracle and/or its affiliates. All rights reserved.

Enter wallet password:
List credential (index: connect_string username)
2: hdal_stby dbaadmin
1: hdal_primary dbaadmin

Add the following in the server's $TNS_ADMIN/sqlnet.ora:

#Point out the server ("client") wallet
WALLET_LOCATION =
  (SOURCE =
    (METHOD = FILE)
    (METHOD_DATA =
  (DIRECTORY = /orasw/product/12201/network/admin/wallet)
  )
 )
 
#This setting causes all CONNECT /@db_connect_string statements to use the information in the wallet at the specified location to authenticate to databases:
SQLNET.WALLET_OVERRIDE = TRUE
 
#BUG 20721271 DUPLICATE FOR STANDBY FROM ACTIVE DATABASE HANGS WHILE RESTORING CONTROL FILE
#Turning this parameter on disables the ability to send and receive "break" messages using urgent data provided by the underlying protocol.
#This would apply to all protocols used by the client.
DISABLE_OOB=on

It works fine with an environmental variable directly in the sqlnet.ora file, too, instead of an absolute path:

WALLET_LOCATION =
  (SOURCE =
    (METHOD = FILE)
    (METHOD_DATA =
  (DIRECTORY = $TNS_ADMIN/wallet)
  )
 )

You should now be able to connect to both databases using rman. Test with a simple script called conntest.cmd.

connect target /@hdal_primary
connect auxiliary /@hdal_stby
run{
allocate channel c1 type disk;
allocate channel c2 type disk;
allocate auxiliary channel aux1 type disk;
allocate auxiliary channel aux2 type disk;
configure device type disk parallelism 2;
}
exit

Output in my case was:

connected to target database: HDAL (DBID=1893290026)

connected to auxiliary database: HDAL (not mounted)

using target database control file instead of recovery catalog
allocated channel: c1
channel c1: SID=1192 device type=DISK

allocated channel: c2
channel c2: SID=52 device type=DISK

allocated channel: aux1
channel aux1: SID=1152 device type=DISK

allocated channel: aux2
channel aux2: SID=1150 device type=DISK

old RMAN configuration parameters:
CONFIGURE DEVICE TYPE DISK PARALLELISM 2 BACKUP TYPE TO BACKUPSET;
new RMAN configuration parameters:
CONFIGURE DEVICE TYPE DISK PARALLELISM 2 BACKUP TYPE TO BACKUPSET;
new RMAN configuration parameters are successfully stored
released channel: c1
released channel: c2
released channel: aux1
released channel: aux2

How to avoid ERROR: role cannot be dropped because some objects depend on it in PostgreSQL

When you try to drop a user in a PostgreSQL database, you may encounter this error:

postgres=# drop role scott;
ERROR:  role "scott" cannot be dropped because some objects depend on it
DETAIL:  privileges for database mydb01

Solution: Drop the user's objects first:

postgres=# drop owned by scott;
DROP OWNED

Dropping can now proceed:

postgres=# drop role scott;
DROP ROLE

If you get errors like:

postgres=# drop role scott;
ERROR:  role "scott" cannot be dropped because some objects depend on it
DETAIL:  privileges for database mydb02
25 objects in database mydb02

You need to open a connection to mydb02 and revoke the privileges, like this:

postgres=# \connect mydb02
You are now connected to database "mydb02" as user "postgres".

Try dropping the role now, and you'll get a list of objects with dependencies (the list is abbriviated):

mydb02=# drop role scott;
ERROR:  role "scott" cannot be dropped because some objects depend on it
DETAIL:  privileges for sequence my_seq1
privileges for sequence my_seq2
privileges for sequence my_seq3
privileges for table mytable1
privileges for table mytable2
privileges for table mytable3
privileges for schema public

Revoke access to all the public tables:

mydb02=# revoke all on all tables in schema public from scott;
REVOKE

When you attempt to drop the role now, you can see that all the tables previously listed are gone:

mydb02=# drop role scott;
ERROR:  role "scott" cannot be dropped because some objects depend on it
DETAIL:  privileges for sequence my_seq1
privileges for sequence my_seq2
privileges for sequence my_seq3
privileges for schema public

Privileges to sequences and a schema remains. Revoke access to the sequences:

mydb02=# revoke all on all sequences in schema public from scott;
REVOKE
mydb02=# drop role scott;
ERROR:  role "scott" cannot be dropped because some objects depend on it
DETAIL:  privileges for schema public

Revoke the final privilege:

mydb02=# revoke usage on schema public from scott;
REVOKE

Finally, we can drop the role:

mydb02=# drop role scott;
DROP ROLE

How to work around hung rman process when duplicating from active database

Applicable for Oracle 12.1.

During a "clone from active database" operation, I had a seemingly stuck restore process. It used a lot of time on restoring the controlfiles.

My clonescript was simple enough:

connect target /@mysrcdb
connect auxiliary /@myauxdb
run{
allocate channel c1 type disk;
allocate channel c2 type disk;
allocate auxiliary channel aux1 type disk;
allocate auxiliary channel aux2 type disk;
configure device type disk parallelism 2;
debug io;
DUPLICATE TARGET DATABASE TO myauxdb
FROM ACTIVE DATABASE
;
debug off;
}
exit

I noticed that the logfile wouldn't move beoynd this point:

DBGANY:     No untilscn in effect [17:44:29.012] (krmkicat)
RMAN-08016: channel aux1: starting datafile backup set restore
RMAN-08169: channel aux1: using network backup set from service mysrcdb
RMAN-08021: channel aux1: restoring control file

Solution:
On the axiliary server, set the parameter

DISABLE_OOB=on

in your $TNS_ADMIN/sqlnet.ora, and reexecute the duplicate-command again.

This is caused by a bug mentioned in Doc ID 2073604.1: "RMAN active duplicate hanging on restore control file" found on support.oracle.com

What is the difference between real-time apply and real-time query?

Real-time apply is the process of applying the redo in the current standby redo log file as it is being filled, in contrast to waiting for the standby redo log to be filled up. By default, apply services will wait for a standby redo log file to be archived, then apply the redo that it contains.


Real-time query is the process of applying redo redo while the physical standby database is open
This allows for read-only usage of the data which is identical to the primary database.
Note: Real-time query is licenced as the "Active Data Guard" option.


Real-time apply is documented here

Real-time query is documented here

How to list index subpartitions and their sizes

SELECT IDXSUBPART.INDEX_OWNER, IDXSUBPART.INDEX_NAME, IDXSUBPART.PARTITION_NAME, IDXSUBPART.SUBPARTITION_NAME,IDXSUBPART.TABLESPACE_NAME, ROUND(SUM(BYTES)/1024/1024/1024) "GB"
FROM   DBA_IND_SUBPARTITIONS IDXSUBPART JOIN DBA_SEGMENTS S
ON (IDXSUBPART.SUBPARTITION_NAME = S.PARTITION_NAME )
WHERE  IDXSUBPART.INDEX_NAME='SALES'
AND    IDXSUBPART.TABLESPACE_NAME='SH'
AND    S.SEGMENT_TYPE='INDEX SUBPARTITION'
GROUP BY IDXSUBPART.INDEX_OWNER, IDXSUBPART.INDEX_NAME, IDXSUBPART.PARTITION_NAME, IDXSUBPART.SUBPARTITION_NAME,IDXSUBPART.TABLESPACE_NAME;

Output:

INDEX_OWNER	INDEX_NAME	PARTITION_NAME	SUBPARTITION_NAME	TABLESPACE_NAME	GB
SH	SALES_IDX_SK1	SYS_P18153	SYS_SUBP14709	DATA1	1
SH	SALES_IDX_SK1	SYS_P18157	SYS_SUBP14831	DATA1	1
SH	SALES_IDX_SK1	SYS_P18158	SYS_SUBP14862	DATA2	25
SH	SALES_IDX_SK1	SYS_P18157	SYS_SUBP14832	DATA1	0
SH	SALES_IDX_SK1	SYS_P18142	SYS_SUBP12409	DATA2	32
SH	SALES_IDX_SK1	SYS_P18139	SYS_SUBP12414	DATA1	1

How do you move an index organized table?

The following index needs to be moved from tablespace DATA1 to tablespace DATA2:

select unique segment_name,segment_type,tablespace_name 
from dba_segments 
where owner='SCOTT'
and tablespace_name = 'DATA1'
;

Result:

SEGMENT_NAME	SEGMENT_TYPE	TABLESPACE_NAME
COUNTRY_ID_PK	INDEX	DATA1

Turns out, the index belongs to an IOT:

select i.owner,i.table_name,i.tablespace_name,i.status, t.iot_type
from dba_indexes i join dba_tables t
on (i.table_name = t.table_name)
and t.table_name ='COUNTRIES'
and t.owner=i.owner
where i.index_name='COUNTRY_ID_PK'
and i.owner='SCOTT';

Result in:

OWNER	TABLE_NAME	TABLESPACE_NAME	STATUS	IOT_TYPE
SCOTT	COUNTRIES	DATA1	VALID	IOT

If you try to move the index COUNTR_ID_PK with the usual "alter index ... rebuild" clause, it will fail with

ORA-28650: Primary index on an IOT cannot be rebuilt

Instead, move the table. Since it's an IOT, the index will be moved automatically with it:

alter table SCOTT.COUNTRIES move tablespace DATA2 ONLINE;

Verify that the IOT was moved by executing the same query as previously:

OWNER	TABLE_NAME	TABLESPACE_NAME	STATUS	IOT_TYPE
SCOTT	COUNTRIES	DATA2	VALID	IOT