[CORALCOOL-1719] Add the 'on delete cascade' option to foreign keys Created:
16/Jun/10 Updated: 09/Sep/14
Status:
Project:
Component/s:
Affects
Version/s:
Fix Version/s:
Open
CORAL and COOL
CORAL
None
Type:
Reporter:
Resolution:
Labels:
Remaining
Estimate:
Time Spent:
Original
Estimate:
Question
Zhen Xie
Unresolved
None
Not Specified
Attachments:
lumiSchema.py
lumiSchema.py
sr115178_deletecascadefk.patch
support115178
External issue
ID:
External issue
URL:
None
Priority:
Assignee:
Votes:
Medium
None (Inactive)
0
Not Specified
Not Specified
http://savannah.cern.ch/support/?115178
Description
Hi,
is there a way to set the delete rule when creating a foreign key with coral?
The default FK created by coral has delete rule set to no action. Sometimes I want to use "delete
cascade". Is it possible with coral?
thanks,
Zhen
Comments
Comment by Andrea Valassi [ 18/Jun/10 ]
Hi Zhen, sorry for not replying before.
No there is no support for 'on delete cascade' yet. We should add it: Alex can you please work on this?
This is just about adding 'ON DELETE CASCADE' when creating/altering the constraint, eg a simple test:
drop table testemp;
drop table testdep;
create table testdep ( depid number );
alter table testdep add constraint testdep_pk primary key ( depid );
create table testemp ( empid number, depid number );
--alter table testemp add constraint testemp_dep_fk foreign key ( depid ) references testdep ( depid );
alter table testemp add constraint testemp_dep_fk foreign key ( depid ) references testdep ( depid ) on delete casc
insert into testdep values ( 1 );
insert into testdep values ( 2 );
insert into testemp values ( 100, 1 );
insert into testemp values ( 101, 1 );
insert into testemp values ( 200, 2 );
insert into testemp values ( 201, 2 );
delete from testdep where depid=1; /* fails if no 'on delete cascade' */
select * from testemp;
I would say that we need the following API changes (I may be forgetting something):
- Add a bool 'onDeleteCascade()' to IForeignKey
- Add a bool argument 'onDeleteCascade' to the two createForeignKey methods in ITableSchemaEditor
Then we need the implementation in all backends and tests.
Thanks to Zhen for the suggestion and Alex for the implementation!
Andrea
Comment by Alexander Kalkhof [ 21/Jun/10 ]
Hi Zhen,
I will start with some implementations. Any Idea how the API changes will be deployed? Or should I add also #i
barriers ...
Alex
Comment by Alexander Kalkhof [ 01/Jul/10 ]
Hi Zhen,
I added the functionality to the OraclePlugin
do we need the same for SQLite?
Alex
Comment by Zhen Xie [ 02/Jul/10 ]
Hi Alex,
I don't think delete on cascade is out of the box for sqlite. Either one uses trigger or emulate the behavior with se
SQL statements.
I personally don't need this in sqlite since I only use sqlite for testing. I simple delete the file. But I think it is goo
backends have the same behavior behind the API. So it is up to you.
thanks and cheers,
Zhen
Comment by Alexander Kalkhof [ 02/Jul/10 ]
Hi Zhen,
thanks for the info.
So I leave the functionality as not implemented at the moment.
My local tests for the OraclePlugin are all fine. Lets wait for the nightly tests...
Alex
Comment by Andrea Valassi [ 23/Jul/10 ]
Hi, going through old messages...
Sorry, I think actually that sqlite 3.6.22 does support on delete cascade. I just tested this:
PRAGMA foreign_keys = ON;
drop table if exists testemp;
drop table if exists testdep;
create table testdep ( depid number, constraint testdep_pk primary key ( depid ) );
--create table testemp ( empid number, depid number, constraint testemp_dep_fk foreign key ( depid ) references
( depid ) ); --default is on delete restrict?
- does not seem to work (it always restricts?)
- Oracle-like behaviour
create table testemp ( empid number, depid number, constraint testemp_dep_fk foreign key ( depid ) references t
depid ) on delete cascade );
insert into testdep values ( 1 );
insert into testdep values ( 2 );
insert into testemp values ( 100, 1 );
insert into testemp values ( 101, 1 );
insert into testemp values ( 200, 2 );
insert into testemp values ( 201, 2 );
insert into testemp values ( 301, 3 ); /* fails (if constraints are enabled) */
delete from testdep where depid=1; /* fails if 'on delete restrict' */
select * from testdep;
select * from testemp;
Please loook at the doc on http://www.sqlite.org/foreignkeys.html
FK support is new, but is now working decently!
So Alex could you please implement it in sqlite? We also need tests of ocurse...
One comment: please make some tests about the default behaviour too. The doc says that default is no action, bu
the impression that default is 'restrict' (ie fail as in Oracle). If the default is already an exception on delete, then i
If instead we must specify on delete restrict, then by default we should add this to all FKs in sqlite (at least to all
ones...).
Thanks
Andrea
Comment by Alexander Kalkhof [ 09/Aug/10 ]
Hi all,
The implementation is already done for Oracle.
I will also add for SQLite. The tests will be the same for both backends then.
Alex
Comment by Alexander Kalkhof [ 10/Aug/10 ]
Hi,
the first test indicates that sqlite can't handle foreign key constraints in a restrict way as it used at the moment.
deleting a row out of a foreign key table doesn't effect it or throw an exception from sqlite.
Alex
Comment by Andrea Valassi [ 10/Aug/10 ]
Hi Alex, thanks for looking at this. I think you need the PRAGMA foreign keys to make this work (in the sqlite
commenad line).
Anyway, to make it work in CORAL, you forst need tomake sure FKs work correctly. Before doing this sr #115
you please work on sr #102114 then (implement FKs in sqlite)?
Thanks
Andrea
Comment by Alexander Kalkhof [ 10/Aug/10 ]
Hi,
adding PRAGMA foreign_keys = ON shows no effect on versions below 3.6.22
on version 3.6.22 i got some errors with the same test:
2010-08-10 15:12:54.378 [ERR] SQLiteStatement::prepare 1 no such table: temp.T0C4_DCASCADE_T1
2010-08-10 15:12:54.495 [ERR] Could not copy data to the temporary table ( CORAL :
"TableDescriptionProxy::alterschema" from "CORAL/RelationalPlugins/sqlite" )
I will check why this happen. This will be continued on sr #102114
Alex
Comment by Alexander Kalkhof [ 11/Aug/10 ]
Hi,
implemented. After fixing and completing sr #102114 this was strait forward.
I committed a version for HEAD at the moment.
@Zhen can you please have a look on both changes
Cheers
Alex
Comment by Alexander Kalkhof [ 24/Aug/10 ]
Hi,
I finished the implementation. Should be OK now. All is tagged as CORAL-preview and CORAL_2_3-patches
@Zhen can you please have a look. This version will be in the next release...
Alex
Comment by Zhen Xie [ 24/Aug/10 ]
Hi Alex,
thanks. Does pyCoral interface need to updated? I tried with pycoral, it complains about
detail.createForeignKey('DETAILSOURCE','LUMISUMMARY_ID','LUMISUMMARY','LUMISUMMARY_I
TypeError: function takes exactly 4 arguments (5 given)
I recompiled these in my local area. ConnectionService CoralCommon CoralKernel OracleAccess PyCoral
RelationalAccess
Cheers,
Zhen
Comment by Andrea Valassi [ 24/Aug/10 ]
Hi Zhen, I let Alex reply on PyCoral.
Not eanyway (again Alex please confirm) that you may need to set some switches to your build, a lot of the code
protected with #ifdef CORAL240DC or similar (ATLAS requests binary compatibility so if these are not enable
API remains the same... we are using this trick instead of making branches that would make maintanenvce more
complex.... hope you manage to build ok with the extra feature?).
Cheers
Andrea
Comment by Alexander Kalkhof [ 25/Aug/10 ]
Hi Zhen,
I will have a look tomorrow. Probably the #ifdef's are missing in PyCoral.
Alex
Comment by Alexander Kalkhof [ 25/Aug/10 ]
Hi Zhen,
It would be a great help if you can post your python test here.
Alex
Comment by Zhen Xie [ 25/Aug/10 ]
attached. But I think it's more likely a problem with my working environment...
(file #15613)
Comment by Zhen Xie [ 25/Aug/10 ]
Attachment lumiSchema.py has been added with description: None
Comment by Alexander Kalkhof [ 25/Aug/10 ]
Hi Zhen,
there was a missing part on the new features in PyCoral
I added those, and it works with your test.
You need to update your cvs and rebuild PyCoral
I hope this will fix your problems
Alex
Comment by Zhen Xie [ 25/Aug/10 ]
Hi Alex,
that fixed the problem. I tested on oracle and sqlite (/afs/cern.ch/sw/lcg/external/sqlite/3.6.22_python2.6/i686-slc
gcc43-opt), 'on delete cascade' option works fine with oracle.
But on sqlite, the row in the child table is not deleted. The test (with delete) attached. When I switch back to non
mode,i.e. detail.createForeignKey('DETAILSOURCE',...,FALSE), I don't see the any constraint violation messa
do the same delete by hand from sqlite3 prompt with PRAGMA foreign_keys = ON;
I see the constraint violation message. So I guess the PRAGMA somehow isn't effective in the SQLiteAccess so
stack.
Cheers,
Zhen
(file #15623)
Comment by Zhen Xie [ 25/Aug/10 ]
Attachment lumiSchema.py has been added with description: None
Comment by Alexander Kalkhof [ 26/Aug/10 ]
Hi Zhen,
I added the insert and delete functions also to my test. For ORACLE I got an error:
2010-08-26 11:43:56.538 [ERR] ORA-01400: cannot insert NULL into
("LCG_CORAL_NIGHTLY"."T0C4_PYCORAL_LUMI"."LUMIVERSION") (Executing statement "INS
INTO LCG_CORAL_NIGHTLY."T0C4_PYCORAL_LUMI" ( "LUMISUMMARY_ID", "RUNNUM",
"CMSLSNUM", "LUMILSNUM", "LUMIVERSION", "DTNORM", "LHCNORM", "INSTLUMI",
"INSTLUMIERROR", "INSTLUMIQUALITY", "CMSALIVE", "STARTORBIT", "NUMORBIT",
"LUMISECTIONQUALITY", "BEAMENERGY", "BEAMSTATUS" ) VALUES ( :"LUMISUMMARY_ID",
:"RUNNUM", :"CMSLSNUM", :"LUMILSNUM", :"LUMIVERSION", :"DTNORM", :"LHCNORM", :"INSTL
:"INSTLUMIERROR", :"INSTLUMIQUALITY", :"CMSALIVE", :"STARTORBIT", :"NUMORBIT",
:"LUMISECTIONQUALITY", :"BEAMENERGY", :"BEAMSTATUS" )")
for SqLite the test was fine.
What version of SQLite you are using.
All versions below 3.6.22 don't work with strict FK's
Alex
Comment by Zhen Xie [ 26/Aug/10 ]
Hi Alex,
In the previous post, I meant I didn't get the error from foreign key constraint violation.
The error you show here is about not null constraint. If you insert not null values, they should be gone.
Cheers,
Zhen
Comment by Alexander Kalkhof [ 26/Aug/10 ]
Hi Zhen,
for me your test is fine for SQLite. I don't get any foreign key violations.
Alex
Comment by Alexander Kalkhof [ 27/Aug/10 ]
Hi Zhen,
I have done some tests.
For the new release we changed the foreign key behavior in SqLite. Now you have to set
CORAL_SQLITE_FOREIGN_KEYS_ON explicitly to activate strict foreign keys.
So I run the test with deactivated foreign keys:
DeleteOnCascade=True Test was fine
DeleteOnCascade=False Test was fine
strict foreign keys:
DeleteOnCascade=True Test was fine
DeleteOnCascade=False Test failed
Can you please repeat and check on on your own
Alex
Comment by Andrea Valassi [ 06/Mar/12 ]
Hi Zhen,
sorry this was never released before.
I am doing some cleanup as this option may be added to CORAL240.
There are essentially two functionalities here:
- specifying the ON DELETE CASCADE ehen creating the FK
- being able to retrieve the onDeleteCascade flag when retrieving the FK info
For what concerns the second feature, retrieving the onDeleteCascade flag, this is essentially NOT IMPLEMEN
any plugin:
- it is not in class OracleAccess::ForeignKeyInfo and is not filled in
OracleAccess::TableDescriptionProxy::refreshForeignKeyInfo
- it is not in struct MySQLAccess::ForeignKeyConstraint and is not filled in
TableSchemaLoader::loadForeignKeyDescriptions
- it is not in class FrontierAccess::ForeignKeyInfo and is not queried via the SQL in
FrontierAccess::TableDescriptionProxy::refreshForeignKeyInfo()
- it is not in class SQLiteAccess::ForeignKeyInfo and is not filled in
SQLiteAccess::TableDescriptionProxy::refreshForeignKeyInfo
- it is not decoded/encoded in the CORAL_SERVER stubs SegmentWriterIterator::append(const TableDescripti
data) and SegmentReaderIterator::extract(TableDescription& data)
So I would say we should concentrate only on the first feature at most. I will give more details later.
Andrea
Comment by Andrea Valassi [ 06/Mar/12 ]
Other problems.
For the second feature (readback):
- onDeleteCascade is not implemented in PyCoral for TableDescription and IForeignKey (it is only implemented
ITableSchemaEditor)
For the first feature:
- the Tests/Integration_DeleteCascade/src/DeleteCascade.cpp test does not test anything specific to setting the
onDeleteConstraints flag
- it is not implemented in MySQLAccess, onDeleteCascade is commented out
void TableSchemaEditor::createForeignKey( const std::string& name, con
std::vector<std::string>& columnNames, const std::string& referencedTableN
const std::vector<std::string>& referencedColumnNames, bool
/*onDeleteCascade*/ )
So it seems that what IS implemented is only
- the addition of the correct SQL flag in OracleAccess and SQLiteAccess
- the API extensions in RelationalAccess (all is there and looks good)
- some (not all) of the internal handling of the onDeleteCascade flag inside the RelationalAccess classes
At this stage there are several options for the immediate future (in view of a CORAL240 release)
- 1. implement the missing functionality: NO, it would take too long before we release CORA240
- 2. keep the API inside CORAL240 (and implement it eventually), so that we can implement it without an API c
- 3. remove the API and all related changes, keep them in a patch, and apply them back, completing the patch, w
necessary
I am tempted to go for option 3. After all, having the AI changes with half an implementation is a maintenance o
for us and a source of confusion for users; and it does not really add much (excpet for ATLAS maybe, who did n
express any interest in this feature). If we do implement this completely at some point, we can always add back t
and al the rest, and for CMS and LHCb it wouldnot be aproblem to rebuild CORAL without binary compatibility
Sorry Zhen this will not be implemented sooner...
Andrea
Comment by Andrea Valassi [ 07/Mar/12 ]
I have removed all code fragments from this incomplete implementation.
The patch used to remove it is
cvs diff -c coral/CoralBase/CoralBase/VersionInfo.h coral/MySQLAccess/src/TableSchemaEditor.cpp
coral/MySQLAccess/src/TableSchemaEditor.h coral/OracleAccess/src/OracleTableBuilder.cpp
coral/OracleAccess/src/TableDescriptionProxy.cpp coral/OracleAccess/src/TableDescriptionProxy.h
coral/PyCoral/src/ITableSchemaEditor.cpp coral/RelationalAccess/RelationalAccess/IForeignKey.h
coral/RelationalAccess/RelationalAccess/ITableSchemaEditor.h
coral/RelationalAccess/RelationalAccess/TableDescription.h coral/RelationalAccess/src/ForeignKey.h
coral/RelationalAccess/src/TableDescription.cpp coral/SQLiteAccess/src/SQLiteTableBuilder.cpp
coral/SQLiteAccess/src/TableDescriptionProxy.cpp coral/SQLiteAccess/src/TableDescriptionProxy.h
coral/config/qmtest/testlist.xml > src/sr115178_deletecascadefk.patch
The commit is the following
/cvs/coral/coral/CoralBase/CoralBase/VersionInfo.h,v <-- coral/CoralBase/CoralBase/VersionInfo.h
new revision: 1.10.2.7; previous revision: 1.10.2.6
/cvs/coral/coral/MySQLAccess/src/TableSchemaEditor.cpp,v <-- coral/MySQLAccess/src/TableSchemaEditor.c
new revision: 1.31; previous revision: 1.30
/cvs/coral/coral/MySQLAccess/src/TableSchemaEditor.h,v <-- coral/MySQLAccess/src/TableSchemaEditor.h
new revision: 1.11; previous revision: 1.10
/cvs/coral/coral/OracleAccess/src/OracleTableBuilder.cpp,v <-- coral/OracleAccess/src/OracleTableBuilder.cpp
new revision: 1.26; previous revision: 1.25
/cvs/coral/coral/OracleAccess/src/TableDescriptionProxy.cpp,v <-- coral/OracleAccess/src/TableDescriptionPro
new revision: 1.58; previous revision: 1.57
/cvs/coral/coral/OracleAccess/src/TableDescriptionProxy.h,v <-- coral/OracleAccess/src/TableDescriptionProxy
new revision: 1.27; previous revision: 1.26
/cvs/coral/coral/PyCoral/src/ITableSchemaEditor.cpp,v <-- coral/PyCoral/src/ITableSchemaEditor.cpp
new revision: 1.38; previous revision: 1.37
/cvs/coral/coral/RelationalAccess/RelationalAccess/IForeignKey.h,v <-coral/RelationalAccess/RelationalAccess/IForeignKey.h
new revision: 1.7; previous revision: 1.6
/cvs/coral/coral/RelationalAccess/RelationalAccess/ITableSchemaEditor.h,v <-coral/RelationalAccess/RelationalAccess/ITableSchemaEditor.h
new revision: 1.16; previous revision: 1.15
/cvs/coral/coral/RelationalAccess/RelationalAccess/TableDescription.h,v <-coral/RelationalAccess/RelationalAccess/TableDescription.h
new revision: 1.16; previous revision: 1.15
/cvs/coral/coral/RelationalAccess/src/ForeignKey.h,v <-- coral/RelationalAccess/src/ForeignKey.h
new revision: 1.8; previous revision: 1.7
/cvs/coral/coral/RelationalAccess/src/TableDescription.cpp,v <-- coral/RelationalAccess/src/TableDescription.cp
new revision: 1.21; previous revision: 1.20
/cvs/coral/coral/SQLiteAccess/src/SQLiteTableBuilder.cpp,v <-- coral/SQLiteAccess/src/SQLiteTableBuilder.cp
new revision: 1.29; previous revision: 1.28
/cvs/coral/coral/SQLiteAccess/src/TableDescriptionProxy.cpp,v <-- coral/SQLiteAccess/src/TableDescriptionPr
new revision: 1.50; previous revision: 1.49
/cvs/coral/coral/SQLiteAccess/src/TableDescriptionProxy.h,v <-- coral/SQLiteAccess/src/TableDescriptionProx
new revision: 1.8; previous revision: 1.7
/cvs/coral/coral/config/qmtest/testlist.xml,v <-- coral/config/qmtest/testlist.xml
new revision: 1.137; previous revision: 1.136
/cvs/coral/coral/config/qmtest/integration.qms/deletecascade_mysql-coral.qmt,v <-coral/config/qmtest/integration.qms/deletecascade_mysql-coral.qmt
new revision: delete; previous revision: 1.9
/cvs/coral/coral/config/qmtest/integration.qms/deletecascade_mysql.qmt,v <-coral/config/qmtest/integration.qms/deletecascade_mysql.qmt
new revision: delete; previous revision: 1.5
/cvs/coral/coral/config/qmtest/integration.qms/deletecascade_oracle-coral.qmt,v <-coral/config/qmtest/integration.qms/deletecascade_oracle-coral.qmt
new revision: delete; previous revision: 1.8
/cvs/coral/coral/config/qmtest/integration.qms/deletecascade_oracle-frontier.qmt,v <-coral/config/qmtest/integration.qms/deletecascade_oracle-frontier.qmt
new revision: delete; previous revision: 1.5
/cvs/coral/coral/config/qmtest/integration.qms/deletecascade_oracle.qmt,v <-coral/config/qmtest/integration.qms/deletecascade_oracle.qmt
new revision: delete; previous revision: 1.5
/cvs/coral/coral/config/qmtest/integration.qms/deletecascade_sqlite.qmt,v <-coral/config/qmtest/integration.qms/deletecascade_sqlite.qmt
new revision: delete; previous revision: 1.5
/cvs/coral/coral/Tests/Integration_DeleteCascade/src/DeleteCascade.cpp,v <-coral/Tests/Integration_DeleteCascade/src/DeleteCascade.cpp
new revision: delete; previous revision: 1.7
/cvs/coral/coral/Tests/Integration_DeleteCascade/src/DeleteCascade.h,v <-coral/Tests/Integration_DeleteCascade/src/DeleteCascade.h
new revision: delete; previous revision: 1.3
/cvs/coral/coral/Tests/Integration_DeleteCascade/src/main.cpp,v <-- coral/Tests/Integration_DeleteCascade/src/m
new revision: delete; previous revision: 1.7
Andrea
Comment by Andrea Valassi [ 07/Mar/12 ]
I am attaching the patch. I keep this open at low priority.
Apologies to Zhen again for not implementing this yet.
Andrea
(file #24387)
Comment by Andrea Valassi [ 07/Mar/12 ]
Attachment sr115178_deletecascadefk.patch has been added with description: None
Comment by Andrea Valassi [ 07/Mar/12 ]
Note that sr #102114 about FK suppot in sqlite is still open too.
Generated at Sun Mar 06 09:57:38 CET 2016 using JIRA 6.4.9#64024sha1:1f1084e06c9893c77549621edbccfecfaa68be5d.