Skip navigation
17517 Views 6 Replies Latest reply: Dec 21, 2011 12:11 PM by nickg RSS
anushree2187 New Enzee 4 posts since
Feb 11, 2010
Currently Being Moderated

Feb 11, 2010 7:30 AM

Primary and Foreign key constraints not enforced

Hi,

 

I have created tables Year,Quarter and Month in my Netezza DB

For thses tables I have :-

                          Year_id--->Primary key in Year table

                          Quarter_Id--->Primary key in Quarter table

                    and Month_Id------> Primary key in Month table

also,

Year_id is foreign key in Quarter table as well as Month table and Quarter_Id is foreign key in Month table.

 

But after creating these three tables I get a NOTICE:-Primary Key Constraints not Enforced

                                                                            and Foreign Key Constraints not Enforced.

 

what does this Notice indicate? Does it mean that the Key's are not working or the constraints are not followed?

 

Thanx,

Anu

  • David Birmingham Active Enzee 426 posts since
    Sep 24, 2007

    Netezza is a data warehouse appliance. Data warehouse best practices are that the database does not enforce pk/fk integrity.The pk/fk integrity check (on a row-by-row basis) is a transactional scenario. Netezza is not a transactional database, in any sense of the words.

     

    In a data warehouse, the enforcement and management of pk/fk is always the domain of the data processing environment. If we delegate this to the database, it would be a violation of VLDB Rule #10 - Never involve the RDBMS in row-by-row data processing. As such, the database can only tell us that we got it wrong. It cannot actually fix the problem (only the data processing application can).

  • Vijay New Enzee 9 posts since
    Jan 5, 2009
    Currently Being Moderated
    2. Feb 23, 2010 12:02 AM (in response to anushree2187)
    Re: Primary and Foreign key constraints not enforced

    In Netezza, PK and FK are for referential integrity only.

    And PK doesnt enforce a unique constraint. ie., even if u hav made a column as a PK, u can hav duplicates in that column

  • David Birmingham Active Enzee 426 posts since
    Sep 24, 2007
    Currently Being Moderated
    3. Feb 23, 2010 12:57 AM (in response to Vijay)
    Re: Primary and Foreign key constraints not enforced

    That's correct, but when you think about it, the RDBMS sits as a passive gateway. It either allows or disallows but does not actively participate in the process of fixing the data. All it can really say is - "I already have one of those, so I'm throwing an error and not allowing you to add another". And that's fine, but in data warehousing, we'd really rather not see the error at all. Because by the time we see the error, it's just too late. We have to take steps to back out what shouldn't be there in the first place. When could we have known this? Prior to loading it.

     

    What we need is a way to remove the duplicates completely so that when the data is inserted, the load will complete without errors. The only positive way to do this is to join the incoming data to the target to find the dupes. We can do this far more efficiently in Netezza because we can make it a co-located join, This is far more efficient than testing for this condition one-row-at-a-time. Since Netezza doesn't apply constraints, we have to do this if we don't want dupes. But we'd have to do it with the RDBMS too, since we don't want the RDBMS involved in checking the data at this level. The RDBMS is constrained to check one-row-at-a-time and thus must violate Rule #10 - Never involve the RDBMS in bulk data processing.

     

    So either way, the ELT/ETL process is required to manage the data into place without involving the less efficient constraint behavior.

  • sampath.k New Enzee 1 posts since
    Dec 21, 2011

    Hi,

     

    In continuation to what you are explaining, let us say the source of data for DWH is an excel file or a text file which does not have any constraints, In that case dont you think the DB of the DWH should impose constraints.

  • Shawn Fox Enzee Exraordinaire 1,326 posts since
    Aug 15, 2006
    Currently Being Moderated
    5. Dec 21, 2011 11:05 AM (in response to sampath.k)
    Re: Primary and Foreign key constraints not enforced

    I don't think introducing an unreliable source changes the problem, you still have to clean the data prior to loading it.  Putting a constraint on a table would prevent you from loading duplicate records but it doesn't tell you what to do with them, it just rejects them.  At some point you have to have a process to deal with the duplicates, so you might as well make it part of your ETL process.

     

    So if your data source is Excel (which is questionable in general, but reality sometimes does get in the way of what we would really like to have), you would load the data into a staging table and then use a stored procedure to validate the data, if there are no constraint violations, the procedure would then load the data into the target table, if there are, it might deal with the violations or just reject them and leave it up to the developer / end user to figure out what needs to be done and try again.

  • nickg Rookie 154 posts since
    Mar 16, 2011
    Currently Being Moderated
    6. Dec 21, 2011 12:11 PM (in response to sampath.k)
    Re: Primary and Foreign key constraints not enforced

    The other thing to consider, given Netezza's architecture, you would take a HUGE performance hit if it had to enforce RI constraints.  There are no indexes, so if you are inserting rows with a dozen FKs, it would need to perform 12 queries against 12 tables every time it inserts a row.  Each query is a table scan and consumes all the spus when it is doing it.  It is rather simple and much, much faster to do it yourself prior to inserting.

    Besides, if you are using surrogate keys, the very process of assigning those keys should be robust enough to ensure RI.

More Like This

  • Retrieving data ...

Bookmarked By (0)

Legend

  • Correct Answers - 4 points
  • Helpful Answers - 2 points