Mittwoch, 22. Juni 2016

How to delete multiple records of a key?

If have come across various projects where especially staging tables had no Oracle enabled business key and not data cleansing process in place. It sorrowfully occurred that we were delivered data with multiple records of the same key characteristics and had to cleanse it manually - after those were loaded further. Cleansing can be done using following SQL skeleton.Be aware that we rely on none of the business key columns contain NULL!

To inspect

with BASE
     as (  select *
             from [TABLE NAME]
            where [FILTER])
   , TUPS
     as (  select [BUSINESS KEY COLUMNS]
             from BASE
         group by [BUSINESS KEY COLUMNS]
           having count(*) > 1)
  select /*+ parallel(4) */
         B.*
    from BASE B
         inner join TUPS on (B.[BUSINESS KEY COLUMNS]) in ((TUPS.[BUSINESS KEY COLUMNS]))
order by B.[BUSINESS KEY COLUMNS];

To delete

delete /*+ parallel(4) */
       from [TABLE NAME]
      where rowid in
               (
                  with NUM_RID
                       as (select rowid as RI
                                , [BUSINESS KEY COLUMNS]
                                , row_number()
                                  over(
                                       partition by [BUSINESS KEY COLUMNS]
                                       order by [SORTER COLUMN] [SORT ORDER]
                                      )
                                     as RN
                             from [TABLE NAME]
                            where [FILTER])
                  select RI
                    from NUM_RID
                   where RN > 1
               );

To terminate


merge into [TABLE NAME] DEST
using (
   select T.rowid
        , row_number()
          over(
                partition by [BUSINESS KEY COLUMNS]
                order by [SORTER COLUMN] [SORT ORDER]
              )
             as RN
     from [TABLE NAME] T
    where 1 = 1
      and [FILTER]
) SRC
on (dest.rowid = src.rowid and src.RN > 1)
when matched then
   update set dest.[VALIDITY TO COLUMN] = TO_DATE('-4701-01-01', 'SYYYY-MM-DD')
            , [MARKER CLAUSE];

1 Kommentar:

  1. One could rewrite the finder select with windowing function as the dedup query. I ran a comparison (on 19c) runs three times each and every comparison showed the windowing being about 40 % slower. Take it with some grains of salt.

    AntwortenLöschen