Saturday, January 27, 2024

Solution to ERRORMSG: Invalid SYSTEM database user credential when running txkPostPDBCreationTasks.pl as a part of an EBS database upgrade

When upgrading an EBS database from version 12c to 19c, you have to convert to the mulitenant architechture.

Oracle supplies script for this entire operation as outlined in Doc ID 580629.1 "Interoperability Notes: Oracle E-Business Suite Release 12.1 with Oracle Database 19c"

When reaching section 6 "Run the post PDB script" you are supposed to run the txkPostPDBCreationTasks.pl script which updates the PDB information.

I found that this script is essential and it must run to completion. It is restartable, so if it fails for some reason, you can fix the root cause and rerun the script.

Here is the error I ran into. The logfile shows it, and it's simple to understand:
==============================
Inside searchFileContents()...
==============================
log_file: /u01/app/oracle/product/19x00_231017/appsutil/log/TXK_POST_PDB_Fri_Jan_26_19_15_34_2024/validate_system_password.log
pattern: ERROR
================
Pattern found...
================
EXIT STATUS: 1
Invalid SYSTEM database user credentials.
LOG FILE: /u01/app/oracle/product/19x00_231017/appsutil/log/TXK_POST_PDB_Fri_Jan_26_19_15_34_2024/validate_system_password.log.
*******FATAL ERROR*******
PROGRAM : (/u01/app/oracle/product/19x00_231017/appsutil/bin/txkPostPDBCreationTasks.pl)
TIME    : Fri Jan 26 19:18:47 2024
FUNCTION: main::validateSystemSchemaCredentials [ Level 1 ]
ERRORMSG: Invalid SYSTEM database user credentials.
*******FATAL ERROR*******
PROGRAM : (/u01/app/oracle/product/19x00_231017/appsutil/bin/txkPostPDBCreationTasks.pl)
TIME    : Fri Jan 26 19:18:47 2024
FUNCTION: main::validateSystemSchemaCredentials [ Level 1 ]
ERRORMSG: Invalid SYSTEM database user credentials.
So the system password was not set correctly. Note that this is the only error in the logfile. All other tests had passed!

Since this is a non-EBS password, it can be set manually with the alter user statement, which I did.

Log on to the cdb:
export ORACLE_SID=cdb
sqlplus / as sysdba
alter user system identified by by mysecretpassword container=all;
Test your connection, preferrably from the apptier. Log on to the PDB, in this case EBSTEST, using the system account:
sqlplus system@EBSTEST
Enter password:

Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
SQL> show user
USER is "SYSTEM"
So the password is now confirmed to be correct.

You can now rerun the script
export ORACLE_SID=cdb

# source the environment
cd $ORACLE_HOME/appsutil
. ./txkSetCfgCDB.env dboraclehome=<your 19c OH path here>

perl $ORACLE_HOME/appsutil/bin/txkPostPDBCreationTasks.pl -dboraclehome=<your 19c OH path here> \
-outdir=<your 19c OH path here>/appsutil/log -cdbsid=cdb -pdbsid=EBSTEST \
-appsuser=<APPSUSER> -dbport=<LISTENING PORT> -servicetype=onpremise


Why is it essential that the script runs to completion, correctly without errors?

First of all, it updates important tables in the APPS schema. During an upgrade to 19c, the UTL_FILE_DIRS are no longer found in the database init.ora file or the spfile. It is saved in the tables v$parameter and v$parameter2.
select name, value from v$parameter where name = 'utl_file_dir';

and 

select name, value from v$parameter where name = 'utl_file_dir';
should yield the same output as the text file found in your $ORACLE_HOME/dbs/EBSTEST_utlfiledir.txt file.

In my case the
$ORACLE_HOME/dbs/EBSTEST_utlfiledir.txt
showed
/sw/oracle/product/temp/EBSTEST
/usr/tmp/EBSTEST
/<oracle_19c_home>/appsutil/outbound/EBSTEST_<myservername>
Second, it creates two environment files in your $ORACLE_HOME that can be used to source the cdb and the PDB environments,respectively. The are called

  • cdb_<yourservername>.env
  • EBSTEST_<yourservername>.env

    Third, it creates your database server's context file, $ORACLE_HOME/appsutil/EBS_<yourservername>.xml

    Without all of the above correctly done, you cannot proceed with running autoconfig on your apptier. It will keep failing until you fix the database-part of your migration.
  • Friday, January 26, 2024

    How to find installed patches through SQL in an EBS database

    During an EBS upgrade, I had to check whether or not a specific patch, 8796558, still needed to be applied.
    Turned out is no longer relevant; it has been rolled up in cummulative patch sets. See listing below.
    set lines 200
    col patch_name format a30
     
    SELECT DISTINCT a.bug_number,e.patch_name, e.patch_type, TRUNC(c.end_date) applied_Date
    FROM ad_bugs a, ad_patch_run_bugs b, ad_patch_runs c, ad_patch_drivers d , ad_applied_patches e
    WHERE
    a.bug_id = b.bug_id AND
    b.patch_run_id = c.patch_run_id AND
    c.patch_driver_id = d.patch_driver_id AND
    d.applied_patch_id = e.applied_patch_id AND
    a.bug_number in
    ('8796558')
    ORDER BY 2 DESC;
     
    BUG_NUMBER                     PATCH_NAME                     PATCH_TYPE                     APPLIED_DATE
    ------------------------------ ------------------------------ ------------------------------ ------------
    8796558                        22644544                       PATCH-SET                      04-SEP-19  
    8796558                        21236633                       PATCH-SET                      27-MAI-16  
    8796558                        19030202                       PATCH-SET                      15-DES-15  
    8796558                        17774755                       PATCH-SET                      11-OKT-14  
     
    22644544 --> Oracle E-Business Suite Release 12.1.3+ Recommended Patch Collection 5 [RPC5]
    21236633 --> Oracle E-Business Suite Release 12.1.3+ Recommended Patch Collection 4 [RPC4]
    19030202 --> Oracle E-Business Suite Release 12.1.3+ Recommended Patch Collection 2 [RPC2]
    17774755 --> Oracle E-Business Suite Release 12.1.3+ Recommended Patch Collection 1 [RPC1]
    

    [DBT-50000] Unable to check available system memory when running dbca

    dbca returned the following error upon launch:
    [FATAL] [DBT-50000] Unable to check available system memory.
       CAUSE: An exception occured while checking kernel parameter.
    *ADDITIONAL INFORMATION:*
    Exception details
     - The default location for framework home is not available. It must be specified
    
    Cause:

    The TMP and/or TMPDIR directory doesn't exist.

    Solution:

    Make sure the .bash_profile contains the correct values for the TMP and TMPDIR directories. It must be an accessible and writable directory.

    In my case:

    mkdir -p /u01/ora19c/tmp
    
    Open .bash_profile, and add the following
    OWNER=ora19c;export OWNER
    TMP=/u01/$OWNER/tmp;export TMP
    TMPDIR=/u01/$OWNER/tmp;export TMPDIR
    
    Launch dbca again. It should now proceed as expected.

    This article from IBM pointed me in the right direction.

    Doc ID 2534894.1 "How To Change Default TEMP Location When Creating Database By DBCA?" is also onto the same solution, but here oracle show you how to append the -j flag to your dbca command, e.g

     dbca -J-Djava.io.tmpdir=/home/oracle/tmp
    

    Friday, January 12, 2024

    Syntax for dropping standby redo logs

    In one of my primary databases, I had several old standby redo log files that were forgotten from a previous relocation using Data Guard. They could all be dropped at this point.

    set lines 200
    col member format a50
    select s.group#,s.thread#,s.sequence#,s.archived,s.status,f.member, f.type
    from v$standby_log s, v$logfile f
    where f.type = 'STANDBY'
    and s.group# = f.group#;
    
        GROUP#    THREAD#  SEQUENCE# ARC STATUS     MEMBER                                             TYPE
    ---------- ---------- ---------- --- ---------- -------------------------------------------------- -------
             6          1          0 YES UNASSIGNED /data03/oradata/prod01/onlinelog/stb_redo06.log         STANDBY
             7          1          0 YES UNASSIGNED /data03/oradata/prod01/onlinelog/stb_redo07.log         STANDBY
             8          1          0 YES UNASSIGNED /data03/oradata/prod01/onlinelog/stb_redo08.log         STANDBY
             9          1          0 YES UNASSIGNED /data03/oradata/prod01/onlinelog/stb_redo09.log         STANDBY
            10          1          0 YES UNASSIGNED /data03/oradata/prod01/onlinelog/stb_redo10.log         STANDBY
            11          1          0 YES UNASSIGNED /data03/oradata/prod01/onlinelog/stb_redo11.log         STANDBY
    
    Syntax for dropping:
    SQL> alter database drop standby logfile group n;
    
    In my case
    alter database drop standby logfile group 6;
    alter database drop standby logfile group 7;
    
    etc etc.
    

    The official Oracle 19c Alter Database documentation is here

    Wednesday, January 3, 2024

    Workaround for error adstpall.sh: too few arguments specified when stopping the EBS 12.2 server processes

    ebs@ebsserver1.oric.no 35642926]$ $ADMIN_SCRIPTS_HOME/adstpall.sh –skipNM -skipAdmin
     
    You are running adstpall.sh version 120.22.12020000.7
     
     
    Enter the WebLogic Server password:
    adstpall.sh: too few arguments specified.
     
    USAGE:  adstpall.sh  [-skipNM] [-skipAdmin] [-nothreading]
            adstpall.sh  -secureapps [-skipNM] [-skipAdmin]
            adstpall.sh -nodbchk [-skipNM] [-skipAdmin]
            adstpall.sh -mode=allnodes
     
     
    adstpall.sh: exiting with status 1
    
    Reason: You have copied the text directly from step 5 in the documentation

    Turns out the docs use a different dash character than the expected one. The one used in the documentation is most likely copied from MS word or similar, which is interpreted differently than the regular dash character:
  • Incorrect: –skipNM
  • Correct: -skipNM
  • In other words, the documented command below is incorrect:
    $ $ADMIN_SCRIPTS_HOME/adstpall.sh skipNM -skipAdmin
    
    Workaround: execute it with the correct dash
    $ $ADMIN_SCRIPTS_HOME/adstpall.sh -skipNM -skipAdmin
    
    Thanks to Marco DeDecker from Oracle Netherlands for finding and pointing out this error.

    Monday, December 11, 2023

    How to write custom messages to the alert log file

    exec sys.dbms_system.ksdwrt(2,'Your message here')
    
    The first argument can be either

  • 1 -> Write to the trace file
  • 2 -> Write to the database alert log file
  • 3 -> Write to both of the above files
  • Friday, December 8, 2023

    Syntax for applying and removing a patch on the WLS server Home in an EBS 12.2 installation

    The bsu (or "Smart update") utility:
    cd $MW_HOME/utils/bsu
    bsu.sh -version
    Oracle Smart Update. Version: 3.3.0.0
    
    Here's an example on how to install a patch:
    ./bsu.sh -remove -patch_download_dir=$MW_HOME/utils/bsu/cache_dir -patchlist=1LRI -prod_dir=$MW_HOME/wlserver_10.3 -verbose -log=install_1LRI.log
    
    Here's an example on how to remove a patch:
    ./bsu.sh -install -patch_download_dir=$MW_HOME/utils/bsu/cache_dir -patchlist=E7HI -prod_dir=$MW_HOME/wlserver_10.3 -verbose -log=install_E7HI.log