↧
SET tables - response (1) by sgarlapa
↧
Calculate Performance Impact of Secondary Index - response (2) by Teradata_SA
Thank you very much for the feedback .
↧
↧
Calculating day number of the year for a date - forum topic by sandy1690
↧
Converting SAS code to Teradata - forum topic by Harpreet Singh
Hello Everyone.
I am looking for tool to convert SAS code to Teradata SQL. Please let me know if something is available in this respect.
Thanks
Harpreet
Forums:
↧
TARA BAM 1300: 3: System Detected Error - response (1) by Jeanne-Anne
I know this is an old post, but it doesn't have any replies.
We got this error when a backup was blocked on the database behind a long running "create table as" statement. NetBackup finally timed out, aborting the backup and leaving a HUT lock on the database which blocked the next "create table" statement until we released the lock.
I have not yet found a way to have ARC skip over tables it can't get a lock on to prevent this problem in the first place.
↧
↧
Stored Procedures and I/O - response (6) by YouriD
sorry for the delay dieter, I added mail-alerts on my thread and - as I didn't get any mails - tought no response had been posted yet.
Hereby the (slightly cleaned-out) stored procedure
/*
CREATED BY : Youri Donders
PRM_PERIOD_YEAR
PRM_PERIOD_MONTH
PRM_TABLE_OUT
Conditions
- All columns (except for filter and sub_period) must contain data
- Output-table must exist before the script is executed
- format of the output-table: main_period, sub_period, KPI_RULE, KPI_Value
*/
Replace Procedure P0_BUS_EUDS_TECV.PRO_BUS_KPI_SUMMARY (PRM_KPI_GROUP VARCHAR(50), PRM_MAIN_PERIOD INTEGER, PRM_SUB_PERIOD INTEGER, PRM_TBL_OUT VARCHAR(60))
Main_proc:
Begin
Declare vcscript varchar(60000);
Declare chkColumn varchar(60);
Declare chkTable varchar(60);
Declare pTableOut varchar(60);
Declare pKPI_Group varchar(500);
/***************************************************************************/
/*********************** Calculation of the KPI ****************************/
/***************************************************************************/
set vcscript = '';
set chkColumn = '';
set pKPI_Group = ',' || PRM_KPI_GROUP || ',';
/* Looping though all KPI's defined in P0_BUS_EUDS_TEC.DSC_PMLT_R_KPI_RULES*/
For acc_cur As cursor1 Cursor For
sel a.*
from P0_BUS_EUDS_TEC.DSC_PMLT_R_KPI_RULES A
where a.KPI_NAME is not null
and a.input_table is not null
and a.kpi_variable is not null
and ( a.kpi_operation is not null or a.kpi_variable like '%.%' )
and a.column_main_period is not null
and :pKPI_Group like '%' || ',' || trim(a.kpi_group) || ',' || '%'
and a.kpi_active = 'Y'
Do
/* building insert-command using data found in the DSC-table */
set vcscript = 'insert into ' || PRM_TBL_OUT || '
(
MAIN_PERIOD
, SUB_PERIOD
, KPI_RULE
, KPI_VALUE
)
select
' || PRM_MAIN_PERIOD || ' as SPROC_MAIN_PERIOD
, ';
if PRM_SUB_PERIOD is null
then set vcscript = vcscript || ' null as SPROC_SUB_PERIOD
';
else set vcscript = vcscript || PRM_SUB_PERIOD || ' as SPROC_SUB_PERIOD
';
end if;
set vcscript = vcscript || ', ' || acc_cur.kpi_rule || ' as SPROC_KPI_RULE
, ';
set vcscript = vcscript || trim(coalesce(acc_cur.kpi_operation,'')) || '(' || trim(acc_cur.kpi_variable) || ') as SPROC_KPI_VALUE
';
set vcscript = vcscript || ' from ' || trim(acc_cur.Input_Table) || ' A
where ';
if PRM_SUB_PERIOD is null
then set vcscript = vcscript || 'a.' || acc_cur.column_main_period || ' = ' || PRM_MAIN_PERIOD || '';
else
/* main period is mandatory, sub period isn't */
if acc_cur.column_sub_period is null
then set vcscript = vcscript || 'a.' || acc_cur.column_main_period || ' = ((' || PRM_MAIN_PERIOD || ' * 100) + ' || PRM_SUB_PERIOD || ')
';
else set vcscript = vcscript || 'a.' || acc_cur.column_main_period || ' = ' || PRM_MAIN_PERIOD || '
and a.' || acc_cur.column_sub_period || ' = ' || PRM_SUB_PERIOD || '' ;
end if;
end if;
/* adding filter condition */
if acc_cur.kpi_filter is not null
then
set vcscript = vcscript || ' and ' || acc_cur.kpi_filter || '';
end if;
set vcscript = vcscript || ' group by SPROC_MAIN_PERIOD
, SPROC_SUB_PERIOD
, SPROC_KPI_RULE
';
if acc_cur.kpi_operation is null
then set vcscript = vcscript || ' , sproc_kpi_value
';
end if;
set vcscript = vcscript || ';';
CALL P0_BUS_EUDS_TECV.TECEXECSQL(:vcScript);
INSERT INTO P0_BUS_EUDS_TEC.FACT_TEC_S_SQL_LOGS VALUES (CURRENT_TIMESTAMP(2),SESSION,USER , 'PRO_BUS_KPI_SUMMARY',:vcScript,:SQLCODE);
end for;
End;
In a small resume:
Theres 1 output-table (which is created before the procudure is started)
In most cases the table contains only 4 columns: main_period, sub_period, kpi_rule and kpi_value with the first 3 in a unique primary index
in the procedure I use a cursor to loop though another table which contains all necessary info for the SQL I need to generate. (1 line per KPI_Rule --> 1 SQL to generate)
After creation of the SQL in a local variable it is executed.
I have the impression that the total IO is finally the IO of statement 1 + the IO of statement 1+2 + the io of statement 1+2+3 etc
Recently I added a lot of "rules" - creating a lot more loops in the cursor - and the IO is now sky-high (and the It-department)
I logged all executed SQL-statements and executed these in a seperate job, but that IO-data is not yet available to me.
I'll post IO and CPU-data tomorrow
Scripts are launched in BTEQ-mode
No deletes are done within the stored procedure.
I hope you're still looking at this topic
regards
Youri
↧
How is the Node Decided - response (5) by krishaneesh
Thanks Dieter for the correction.
Krishna
↧
Calculating day number of the year for a date - response (1) by krishaneesh
sel calendar_date,day_of_year from sys_calendar.calendar
This gives the day of the year
↧
Calculating day number of the year for a date - response (2) by krishaneesh
sel (case (current_date mod 10000)/100 when 1 then current_date mod 100 when 2 then current_date mod 100 + 31 when 3 then current_date mod 100 + 59 when 4 then current_date mod 100 + 90 when 5 then current_date mod 100 + 120 when 6 then current_date mod 100 + 151 when 7 then current_date mod 100 + 181 when 8 then current_date mod 100 + 212 when 9 then current_date mod 100 + 243 when 10 then current_date mod 100 + 273 when 11 then current_date mod 100 + 304 when 12 then current_date mod 100 + 334 end) + (case when ((((current_date / 10000 + 1900) mod 4 = 0) AND ((current_date / 10000 + 1900) mod 100 <> 0)) OR ((current_date / 10000 + 1900) mod 400 = 0)) AND ((current_date mod 10000)/100 > 2) then 1 else 0 end)
This will work only for dates from 1900-01-01 to 2100-12-31. This is an extension of the above
↧
↧
Stored Procedures and I/O - response (7) by YouriD
As promised the IO and CPU data
I get the numbers from teraqueue.history
Stored Procedure
IO: 7.253.180.525
CPU: 1.081.842
Logged SQL in a seperate script
IO: 64.015.848
CPU: 8.957
--> IO and CPU consumed of SQL executed in batch is only 1% of the measures taken after the stored procedure
↧
Calculating day number of the year for a date - response (3) by YouriD
slightly shorter calculation (in case you don't want to use the calendar-table):
sel date - cast(trim(extract(year from date)) || '/01/01' as date format 'YYYY/MM/DD') + 1
I'm calculating the number of days between the wanted date and the first day of the year of that date
+ 1 because the first day of the year needs to be counted as well
regards
Youri
↧
CDC - forum topic by Raj_k
Hi all,
What is ment by CDC(Change Data Capture) ?? How it is used ??
Thanks
Raj
Forums:
↧
Calculating day number of the year for a date - response (5) by VandeBergB
Youri,
Keep it simple, use the sys_calendar.calendar view. It will keep you covered for the the next 87 years, and the first day of the year is already numbered one so you don't even have to do the math of adding 1.
keep it simple, the code you have to maintain may be your own.
Cheers.
↧
↧
CDC - response (1) by VandeBergB
CDC is a broad term that applies to any of several methods of detecting only the data that changed for loading to avoid row collisions, maintain history and make ELT/ETL more efficient
Cheers
↧
CDC - response (2) by Raja_KT
CDC is change data capture. It is essential in DWH/BI project to see changed data.Different techniques are available like change in timestamps, versioning of rows, indicators, or both timestamps and indicators, messages and queues,logs, events.
Imagine a case when when you pull or push huge source data that have been processed before into downstream edw or applications not neccesary by business.It simply eats up time and effort(example). So keep track only data that is changed and proceed further.
There are many CDC tools available in the market.
Cheers,
↧
Sequence number being resetted - forum topic by KVB
Hi all,
I have a table with one of the columns using sequence generator.It is being resetted after somepoint of time.below is the identity column.Is there anythign wrong with the declaration.Though I mentioned CYCLE it should repeat after the end of the maximum number.
CT ABC
(
ID INTEGER NOT NULL GENERATED BY DEFAULT AS IDENTITY
(START WITH 100000000
INCREMENT BY 1
MINVALUE 100000000
MAXVALUE 2147483647
CYCLE)
);
Regards
KVB
Forums:
↧
Sequence number being resetted - response (1) by YouriD
I use the following:
CT ABC ( ID INTEGER GENERATED ALWAYS AS IDENTITY (START WITH 1 INCREMENT BY 1 MINVALUE 1 MAXVALUE 100000000 NO CYCLE) );
It creates a unique value, but not a sequential number
Muliple inserts in the table are possible without having duplicates on ID
regards
Youri
↧
↧
How to reduce skew factor on stage tables with NoPI? - response (6) by manju2399
Hi,
I have a table in PRON which is having 3 Billion data. There are 4 Primary Index column, Partition on Date and secondary Index. when I checked the Distint Primary Indix on the combination of 4 columns. it is aboly 2.98 Billion. Percentage of duplicates is about 2%.
But the Skew in the table is 54. Can you plese explain how reduse the Skew factor.
Appriciate your help.
Thanks,
Manju
↧
Calculating day number of the year for a date - response (6) by Fred
In TD14.0 and later, use td_day_of_year() function
↧
How to reduce skew factor on stage tables with NoPI? - response (7) by Harpreet Singh
Hi Manju,
Secondary index will not impact skew factor.
Skew factor can be reduced by choosing more evenly distributed set of columns. if your 2% is going to one amp then you can find out what distinguishs it in other columns from each other.
But reducing skew factor by choosing more appropriate Primary index column set will not help Teradata performance unless it is used in SQLs. Primary index needs to be mostly usable in sqls as whole and not partially and nearly evenly distributed.
Thanks
Harpreet
↧
you can load 5 tables with 32 NUSIs each using ML.
Please see the Dieter response at below link to similar questions-
http://forums.teradata.com/forum/analytics/why-fastload-does-not-support-nusi#comment-131727
Regards,
Sri