1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
On New Primary
DGMGRL>
reinstate database w5005pr;
Reinstating database "w5005pr", please wait...
Error: ORA-16653: failed to reinstate database
Failed.
Reinstatement of database "w5005pr" failed
DGMGRL>
/*
Failing it would have needed to configure flashback database to be able to do so automatically…
If you try to start w5005pr database in MOUNT state and issue again a request to reinstate you will now get clear error message:
*/
On old Primary (W5005PR)
[oracle@rac1 ~]$ sqlplus / as sysdba
SQL*Plus: Release 11.2.0.3.0 Production on Tue Jan 12 00:05:33 2016
Copyright (c) 1982, 2011, Oracle. All rights reserved.
Connected to an idle instance.
SQL>
startup mount;
ORACLE instance started.
Total System Global Area 1269366784 bytes
Fixed Size 2227984 bytes
Variable Size 1090519280 bytes
Database Buffers 167772160 bytes
Redo Buffers 8847360 bytes
Database mounted.
SQL>
On New Primary
DGMGRL>
reinstate database w5005pr; <---
Reinstating database "w5005pr", please wait...
Operation requires shutdown of instance "w5005pr" on database "w5005pr"
Shutting down instance "w5005pr"...
ORA-01109: database not open
Database dismounted.
ORACLE instance shut down.
Operation requires startup of instance "w5005pr" on database "w5005pr"
Starting instance "w5005pr"...
ORACLE instance started.
Database mounted.
Continuing to reinstate database "w5005pr" ...
Reinstatement of database "w5005pr" succeeded <----
DGMGRL>
Database alert log. (w5005pr)
/*
Starting Data Guard Broker (DMON)
Thu Jan 21 21:43:59 2016
INSV started with pid=27, OS id=6547
Thu Jan 21 21:44:02 2016
NSV1 started with pid=28, OS id=6559
Thu Jan 21 21:44:07 2016
NSV1 started with pid=28, OS id=6567
Thu Jan 21 21:44:10 2016
RSM0 started with pid=30, OS id=6571
Thu Jan 21 21:44:10 2016
Using STANDBY_ARCHIVE_DEST parameter default value as +FRA
RFS[1]: Assigned to RFS process 6575
RFS[1]: Database mount ID mismatch [0xf7edabfc:0xf7ede817] (4159548412:4159563799)
RFS[1]: Not using real application clusters
Thu Jan 21 21:44:12 2016
RFS[2]: Assigned to RFS process 6579
RFS[2]: Database mount ID mismatch [0xf7edabfc:0xf7ede817] (4159548412:4159563799)
RFS[2]: Not using real application clusters
FLASHBACK DATABASE TO SCN 1582275
SUCCESS: diskgroup FRA was mounted
Flashback Restore Start
Thu Jan 21 21:44:13 2016
NOTE: dependency between database w5005pr and diskgroup resource ora.FRA.dg is established
Flashback Restore Complete
Flashback Media Recovery Start
Serial Media Recovery started
Flashback Media Recovery Log +FRA/w5005pr/archivelog/2016_01_21/thread_1_seq_21.421.901747617
Flashback Media Recovery Log +FRA/w5005pr/archivelog/2016_01_21/thread_1_seq_22.422.901747623
Recovery of Online Redo Log: Thread 1 Group 1 Seq 23 Reading mem 0
Mem# 0: +DATA/w5005pr/onlinelog/group_1.278.900894305
Recovery of Online Redo Log: Thread 1 Group 2 Seq 24 Reading mem 0
Mem# 0: +DATA/w5005pr/onlinelog/group_2.279.900894307
Incomplete Recovery applied until change 1582276 time 01/21/2016 21:28:19
Flashback Media Recovery Complete
Completed: FLASHBACK DATABASE TO SCN 1582275
alter database convert to physical standby
ALTER DATABASE CONVERT TO PHYSICAL STANDBY (w5005pr)
Flush standby redo logfile failed:1649
Clearing standby activation ID 4158708098 (0xf7e0d982)
The primary database controlfile was created using the
'MAXLOGFILES 16' clause.
There is space for up to 13 standby redo logfiles
Use the following SQL commands on the standby database to create
standby redo logfiles that match the primary database:
ALTER DATABASE ADD STANDBY LOGFILE 'srl1.f' SIZE 52428800;
ALTER DATABASE ADD STANDBY LOGFILE 'srl2.f' SIZE 52428800;
ALTER DATABASE ADD STANDBY LOGFILE 'srl3.f' SIZE 52428800;
ALTER DATABASE ADD STANDBY LOGFILE 'srl4.f' SIZE 52428800;
Waiting for all non-current ORLs to be archived...
All non-current ORLs have been archived.
Clearing online redo logfile 1 +DATA/w5005pr/onlinelog/group_1.278.900894305
Clearing online log 1 of thread 1 sequence number 23
Clearing online redo logfile 1 complete
Clearing online redo logfile 2 +DATA/w5005pr/onlinelog/group_2.279.900894307
Clearing online log 2 of thread 1 sequence number 24
Clearing online redo logfile 2 complete
Clearing online redo logfile 3 +DATA/w5005pr/onlinelog/group_3.280.900894309
Clearing online log 3 of thread 1 sequence number 25
Clearing online redo logfile 3 complete
Completed: alter database convert to physical standby
Thu Jan 21 21:44:16 2016
Shutting down instance (immediate)
Shutting down instance: further logons disabled
Stopping background process MMNL
Stopping background process MMON
License high water mark = 3
All dispatchers and shared servers shutdown
alter database CLOSE NORMAL
ORA-1109 signalled during: alter database CLOSE NORMAL...
alter database DISMOUNT
Shutting down archive processes
Archiving is disabled
Thu Jan 21 21:44:17 2016
NOTE: Deferred communication with ASM instance
NOTE: deferred map free for map id 4
Thu Jan 21 21:44:17 2016
NOTE: Deferred communication with ASM instance
NOTE: deferred map free for map id 77
Thu Jan 21 21:44:17 2016
NOTE: Deferred communication with ASM instance
NOTE: deferred map free for map id 2
Thu Jan 21 21:44:17 2016
NOTE: Deferred communication with ASM instance
Completed: alter database DISMOUNT
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Shutting down Data Guard Broker processes
Thu Jan 21 21:44:19 2016
Completed: Data Guard Broker shutdown
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Thu Jan 21 21:44:21 2016
Stopping background process VKTM
Thu Jan 21 21:44:21 2016
NOTE: Shutting down MARK background process
Thu Jan 21 21:44:23 2016
Instance shutdown complete
Thu Jan 21 21:44:23 2016
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Picked latch-free SCN scheme 3
WARNING: db_recovery_file_dest is same as db_create_file_dest
Autotune of undo retention is turned on.
IMODE=BR
ILAT =27
LICENSE_MAX_USERS = 0
SYS auditing is disabled
Starting up:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options.
ORACLE_HOME = /u01/app/oracle/product/11.2.0/db_1
System name: Linux
Node name: rac1.rajasekhar.com
Release: 2.6.32-200.13.1.el5uek
Version: #1 SMP Wed Jul 27 21:02:33 EDT 2011
Machine: x86_64
Using parameter settings in server-side pfile /u01/app/oracle/product/11.2.0/db_1/dbs/initw5005pr.ora
System parameters with non-default values:
processes = 150
spfile = "+DATA/w5005pr/spfilew5005pr.ora"
memory_target = 1216M
control_files = "+DATA/w5005pr/controlfile/current.268.900456457"
db_block_size = 8192
compatible = "11.2.0.0.0"
log_archive_dest_1 = "LOCATION=+FRA VALID_FOR=(ALL_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=w5005pr"
log_archive_dest_2 = "service="w5005prg""
log_archive_dest_2 = "LGWR ASYNC NOAFFIRM delay=0 optional compression=disable max_failure=0 max_connections=1 reopen=300 db_unique_name="w5005prg" net_timeout=30"
log_archive_dest_2 = "valid_for=(all_logfiles,primary_role)"
log_archive_dest_state_1 = "ENABLE"
log_archive_dest_state_2 = "ENABLE"
log_archive_min_succeed_dest= 1
fal_client = "W5005PR"
fal_server = "w5005prg"
log_archive_trace = 0
log_archive_config = "DG_CONFIG=(w5005pr,w5005prg)"
log_archive_format = "%t_%s_%r.dbf"
log_archive_max_processes= 30
archive_lag_target = 0
db_create_file_dest = "+DATA"
db_recovery_file_dest = "+DATA"
db_recovery_file_dest_size= 2G
standby_file_management = "AUTO"
undo_tablespace = "UNDOTBS1"
remote_login_passwordfile= "EXCLUSIVE"
db_domain = ""
dispatchers = "(PROTOCOL=TCP) (SERVICE=w5005prXDB)"
job_queue_processes = 0
audit_file_dest = "/u01/app/oracle/admin/w5005pr/adump"
audit_trail = "DB"
db_name = "w5005pr"
db_unique_name = "w5005pr"
open_cursors = 300
dg_broker_start = TRUE
diagnostic_dest = "/u01/app/oracle"
Thu Jan 21 21:44:24 2016
PMON started with pid=2, OS id=6641
Thu Jan 21 21:44:24 2016
PSP0 started with pid=3, OS id=6645
Thu Jan 21 21:44:25 2016
VKTM started with pid=4, OS id=6649 at elevated priority
VKTM running at (1)millisec precision with DBRM quantum (100)ms
Thu Jan 21 21:44:25 2016
...
...
NOTE: initiating MARK startup
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
Starting background process MARK
Thu Jan 21 21:44:25 2016
MMNL started with pid=18, OS id=6711
Thu Jan 21 21:44:25 2016
MARK started with pid=19, OS id=6715
NOTE: MARK has subscribed
starting up 1 shared server(s) ...
ORACLE_BASE not set in environment. It is recommended
that ORACLE_BASE be set in the environment
Reusing ORACLE_BASE from an earlier startup = /u01/app/oracle
Thu Jan 21 21:44:25 2016
DMON started with pid=23, OS id=6736
Thu Jan 21 21:44:25 2016
alter database mount
Thu Jan 21 21:44:25 2016
ALTER SYSTEM SET local_listener='(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.2.101)(PORT=1521))))' SCOPE=MEMORY SID='w5005pr';
NOTE: Loaded library: System
SUCCESS: diskgroup DATA was mounted
NOTE: dependency between database w5005pr and diskgroup resource ora.DATA.dg is established
ARCH: STARTING ARCH PROCESSES
Thu Jan 21 21:44:32 2016
ARC0 started with pid=26, OS id=6786
ARC0: Archival started
ARCH: STARTING ARCH PROCESSES COMPLETE
ARC0: STARTING ARCH PROCESSES
Thu Jan 21 21:44:33 2016
ARC1 started with pid=27, OS id=6790
Successful mount of redo thread 1, with mount id 4159536441
Allocated 4194304 bytes in shared pool for flashback generation buffer
Starting background process RVWR
..
..
ARC0: STARTING ARCH PROCESSES COMPLETE
Thu Jan 21 21:44:38 2016
Starting Data Guard Broker (DMON)
Thu Jan 21 21:44:38 2016
INSV started with pid=64, OS id=6983
Thu Jan 21 21:44:43 2016
NSV1 started with pid=65, OS id=6991
Thu Jan 21 21:44:46 2016
RSM0 started with pid=66, OS id=7001
Thu Jan 21 21:44:46 2016
Using STANDBY_ARCHIVE_DEST parameter default value as +FRA
Thu Jan 21 21:44:46 2016
SUCCESS: diskgroup FRA was mounted
RFS[1]: Assigned to RFS process 7007
RFS[1]: Opened log for thread 1 sequence 25 dbid -136715127 branch 900893050
Archived Log entry 39 added for thread 1 sequence 25 rlc 900893050 ID 0xf7e0d982 dest 2:
Thu Jan 21 21:44:46 2016
NOTE: dependency between database w5005pr and diskgroup resource ora.FRA.dg is established
Thu Jan 21 21:44:46 2016
RFS[2]: Assigned to RFS process 7011
RFS[2]: Opened log for thread 1 sequence 1 dbid -136715127 branch 901748007
RFS[2]: New Archival REDO Branch(resetlogs_id): 901748007 Prior: 900893050
RFS[2]: Archival Activation ID: 0xf7edabfc Current: 0x0
RFS[2]: Effect of primary database OPEN RESETLOGS
RFS[2]: Incarnation entry added for Branch(resetlogs_id): 901748007 (w5005pr)
Thu Jan 21 21:44:46 2016
RFS[3]: Assigned to RFS process 7013
RFS[3]: Opened log for thread 1 sequence 2 dbid -136715127 branch 901748007
Thu Jan 21 21:44:46 2016
Setting recovery target incarnation to 4
Archived Log entry 40 added for thread 1 sequence 1 rlc 901748007 ID 0xf7edabfc dest 2:
Thu Jan 21 21:44:46 2016
RFS[4]: Assigned to RFS process 7015
RFS[4]: Opened log for thread 1 sequence 3 dbid -136715127 branch 901748007
Archived Log entry 41 added for thread 1 sequence 2 rlc 901748007 ID 0xf7edabfc dest 2:
Archived Log entry 42 added for thread 1 sequence 3 rlc 901748007 ID 0xf7edabfc dest 2:
RFS[2]: Selected log 4 for thread 1 sequence 4 dbid -136715127 branch 901748007
Thu Jan 21 21:44:47 2016
Archived Log entry 43 added for thread 1 sequence 4 ID 0xf7edabfc dest 1:
Thu Jan 21 21:44:47 2016
Primary database is in MAXIMUM PERFORMANCE mode
RFS[5]: Assigned to RFS process 7023
RFS[5]: Selected log 4 for thread 1 sequence 5 dbid -136715127 branch 901748007
Data Guard: Failover target was Real Time Query standby; attempting to open this standby after reinstatement ...
ALTER DATABASE OPEN READ ONLY
Data Guard Broker initializing...
Data Guard Broker initialization complete
AUDIT_TRAIL initialization parameter is changed to OS, as DB is NOT compatible for database opened with read-only access
Thu Jan 21 21:44:49 2016
SMON: enabling cache recovery
Dictionary check beginning
Dictionary check complete
Database Characterset is WE8MSWIN1252
No Resource Manager plan active
replication_dependency_tracking turned off (no async multimaster replication found)
Physical standby database opened for read only access.
Completed: ALTER DATABASE OPEN READ ONLY
Thu Jan 21 21:44:50 2016
db_recovery_file_dest_size of 2048 MB is 5.52% used. This is a
user-specified limit on the amount of space that will be used by this
database for recovery-related files, and does not reflect the amount of
space available in the underlying filesystem or ASM diskgroup.
ALTER SYSTEM SET log_archive_trace=0 SCOPE=BOTH SID='w5005pr';
ALTER SYSTEM SET log_archive_format='%t_%s_%r.dbf' SCOPE=SPFILE SID='w5005pr';
ALTER SYSTEM SET standby_file_management='AUTO' SCOPE=BOTH SID='*';
ALTER SYSTEM SET archive_lag_target=0 SCOPE=BOTH SID='*';
ALTER SYSTEM SET log_archive_max_processes=30 SCOPE=BOTH SID='*';
ALTER SYSTEM SET log_archive_min_succeed_dest=1 SCOPE=BOTH SID='*';
ALTER SYSTEM SET fal_server='w5005prg' SCOPE=BOTH;
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE THROUGH ALL SWITCHOVER DISCONNECT USING CURRENT LOGFILE
Attempt to start background Managed Standby Recovery process (w5005pr)
Thu Jan 21 21:44:51 2016
MRP0 started with pid=73, OS id=7050
MRP0: Background Managed Standby Recovery process started (w5005pr)
Serial Media Recovery started
Managed Standby Recovery starting Real Time Apply
Media Recovery start incarnation depth : 1, target inc# : 4, irscn : 1582299
Waiting for all non-current ORLs to be archived...
All non-current ORLs have been archived.
Media Recovery Log +FRA/w5005pr/archivelog/2016_01_21/thread_1_seq_25.425.901748687
Identified End-Of-Redo (failover) for thread 1 sequence 25 at SCN 0x0.1824db
Resetting standby activation ID 4158708098 (0xf7e0d982)
Media Recovery End-Of-Redo indicator encountered
Media Recovery Continuing
Media Recovery Log +FRA/w5005pr/archivelog/2016_01_21/thread_1_seq_1.426.901748687
Media Recovery Log +FRA/w5005pr/archivelog/2016_01_21/thread_1_seq_2.427.901748687
Media Recovery Log +FRA/w5005pr/archivelog/2016_01_21/thread_1_seq_3.428.901748687
Media Recovery Log +FRA/w5005pr/archivelog/2016_01_21/thread_1_seq_4.429.901748687
Media Recovery Waiting for thread 1 sequence 5 (in transit)
Thu Jan 21 21:44:57 2016
Completed: ALTER DATABASE RECOVER MANAGED STANDBY DATABASE THROUGH ALL SWITCHOVER DISCONNECT USING CURRENT LOGFILE
Recovery of Online Redo Log: Thread 1 Group 4 Seq 5 Reading mem 0
Mem# 0: +DATA/w5005pr/onlinelog/group_4.281.900894309
Thu Jan 21 21:45:00 2016
Archived Log entry 44 added for thread 1 sequence 5 ID 0xf7edabfc dest 1:
Media Recovery Waiting for thread 1 sequence 6
Thu Jan 21 21:45:00 2016
Primary database is in MAXIMUM PERFORMANCE mode
RFS[6]: Assigned to RFS process 7054
RFS[6]: Selected log 4 for thread 1 sequence 6 dbid -136715127 branch 901748007
Recovery of Online Redo Log: Thread 1 Group 4 Seq 6 Reading mem 0
Mem# 0: +DATA/w5005pr/onlinelog/group_4.281.900894309
*/
DGMGRL>
show configuration;
Configuration - w5005pr
Protection Mode: MaxPerformance
Databases:
w5005prg - Primary database
w5005pr - Physical standby database
Fast-Start Failover: DISABLED
Configuration Status:
SUCCESS
DGMGRL>