[CRM-15470] MySQL transaction deadlocks with multiple simultaneous
recurring contribution updates Created: 15/Oct/14 Updated: 11/Oct/15 Resolved: 18/Feb/15
Status:
Project:
Component/s:
Affects
Version/s:
Fix Version/s:
Closed
CiviCRM
CiviContribute
4.4.7
Type:
Reporter:
Resolution:
Labels:
Remaining
Estimate:
Time Spent:
Original
Estimate:
Bug
Joseph Lacey
Fixed/Completed
None
Not Specified
Issue Links:
Duplicate
duplicates
Documentation
Required?:
4.5.7
Priority:
Assignee:
Votes:
Minor
Joseph Lacey
0
Not Specified
Not Specified
CRM-15833
No Deadlock Handling
Closed
None
Description
When Authorize.net (not sure about other payment processors) sends more than a few (our current
case is 11) silent updates back to CiviCRM in quick succession when a recurring payment has
been processed, consistently at least two of them don't get recorded in CiviCRM due to a MySQL
transaction deadlock. It isn't always the same two recurring subscriptions.
--------------------------Relevant CiviCRM log output:
Sep 17 04:50:22 [info] Contribution record updated successfully
Sep 17 04:50:22 [info] Success: Database updated
Sep 17 04:52:40 [info] Subscription payment failed - 'The credit card has expired.'
Sep 17 04:52:40 [info] Contribution record updated successfully
Sep 17 04:52:40 [info] Success: Database updated
Sep 17 04:52:40 [info] Contribution record updated successfully
Sep 17 04:52:40 [info] Success: Database updated
Sep 17 04:52:41 [info] Contribution record updated successfully
Sep 17 04:52:41 [info] Success: Database updated
Sep 17 04:52:41 [info] $Fatal Error Details = Array
(
[callback] => Array
(
[0] => CRM_Core_Error
[1] => handle
)
[code] => -1
[message] => DB Error: unknown error
[mode] => 16
[debug_info] => INSERT INTO civicrm_activity_contact (activity_id , contact_id ,
record_type_id ) VALUES ( 1144 , 8948 , 2 ) [nativecode=1213 ** Deadlock found when trying
to get lock; try restarting transaction]
[type] => DB_Error
[user_info] => INSERT INTO civicrm_activity_contact (activity_id , contact_id , record_type_id )
VALUES ( 1144 , 8948 , 2 ) [nativecode=1213 ** Deadlock found when trying to get lock; try
restarting transaction]
[to_string] => [db_error: message="DB Error: unknown error" code=-1 mode=callback
callback=CRM_Core_Error::handle prefix="" info="INSERT INTO civicrm_activity_contact
(activity_id , contact_id , record_type_id ) VALUES ( 1144 , 8948 , 2 ) [nativecode=1213 **
Deadlock found when trying to get lock; try restarting transaction]"]
)
Sep 17 04:52:41 [info] $backTrace = #0
/var/www/example.org/sites/all/modules/civicrm/CRM/Core/Error.php(197):
CRM_Core_Error::backtrace("backTrace", TRUE)
#1 [internal function](): CRM_Core_Error::handle(Object(DB_Error))
#2 /var/www/example.org/sites/all/modules/civicrm/packages/PEAR.php(931):
call_user_func((Array:2), Object(DB_Error))
#3 /var/www/example.org/sites/all/modules/civicrm/packages/DB.php(969): PEAR_Error>PEAR_Error("DB Error: unknown error", -1, 16, (Array:2), "INSERT INTO
civicrm_activity_contact (activity_id , contact_id , record_type_...")
#4 /var/www/example.org/sites/all/modules/civicrm/packages/PEAR.php(564): DB_Error-
>DB_Error(-1, 16, (Array:2), "INSERT INTO civicrm_activity_contact (activity_id , contact_id ,
record_type_...")
#5 /var/www/example.org/sites/all/modules/civicrm/packages/DB/common.php(1905): PEAR>raiseError(NULL, -1, NULL, NULL, "INSERT INTO civicrm_activity_contact (activity_id,
contact_id , record_type_...", "DB_Error", TRUE)
#6 /var/www/example.org/sites/all/modules/civicrm/packages/DB/mysql.php(898): DB_common>raiseError(-1, NULL, NULL, NULL, "1213 ** Deadlock found when trying to get lock; try
restarting transaction")
#7 /var/www/example.org/sites/all/modules/civicrm/packages/DB/mysql.php(327): DB_mysql>mysqlRaiseError()
#8 /var/www/example.org/sites/all/modules/civicrm/packages/DB/common.php(1216):
DB_mysql->simpleQuery("INSERT INTO civicrm_activity_contact (activity_id , contact_id ,
record_type_...")
#9 /var/www/example.org/sites/all/modules/civicrm/packages/DB/DataObject.php(2421):
DB_common->query("INSERT INTO civicrm_activity_contact (activity_id , contact_id ,
record_type_...")
#10 /var/www/example.org/sites/all/modules/civicrm/packages/DB/DataObject.php(1055):
DB_DataObject->query("INSERT INTO civicrm_activity_contact (activity_id , contact_id ,
record_type...")
#11 /var/www/example.org/sites/all/modules/civicrm/CRM/Core/DAO.php(278):
DB_DataObject->insert()
#12
/var/www/example.org/sites/all/modules/civicrm/CRM/Activity/BAO/ActivityContact.php(63):
CRM_Core_DAO->save()
#13 /var/www/example.org/sites/all/modules/civicrm/CRM/Activity/BAO/Activity.php(364):
CRM_Activity_BAO_ActivityContact::create((Array:3))
#14 /var/www/example.org/sites/all/modules/civicrm/CRM/Activity/BAO/Activity.php(1859):
CRM_Activity_BAO_Activity::create((Array:10))
#15 /var/www/example.org/sites/all/modules/civicrm/CRM/Core/Payment/BaseIPN.php(586):
CRM_Activity_BAO_Activity::addActivity(Object(CRM_Contribute_BAO_Contribution),
NULL, NULL)
#16
/var/www/example.org/sites/all/modules/civicrm/CRM/Core/Payment/AuthorizeNetIPN.php(183):
CRM_Core_Payment_BaseIPN->completeTransaction((Array:28), (Array:6), (Array:8),
Object(CRM_Core_Transaction), Object(CRM_Contribute_BAO_ContributionRecur))
#17
/var/www/example.org/sites/all/modules/civicrm/CRM/Core/Payment/AuthorizeNetIPN.php(72):
CRM_Core_Payment_AuthorizeNetIPN->recur((Array:28), (Array:6), (Array:8), FALSE)
#18 /var/www/example.org/sites/all/modules/civicrm/extern/authorizeIPN.php(40):
CRM_Core_Payment_AuthorizeNetIPN->main()
#19
{main}
Sep 17 04:52:41 [info] $Fatal Error Details = Array
(
[callback] => Array
(
[0] => CRM_Core_Error
[1] => handle
)
[code] => -1
[message] => DB Error: unknown error
[mode] => 16
[debug_info] => INSERT INTO civicrm_activity_contact (activity_id , contact_id ,
record_type_id ) VALUES ( 1145 , 8954 , 2 ) [nativecode=1213 ** Deadlock found when trying
to get lock; try restarting transaction]
[type] => DB_Error
[user_info] => INSERT INTO civicrm_activity_contact (activity_id , contact_id , record_type_id )
VALUES ( 1145 , 8954 , 2 ) [nativecode=1213 ** Deadlock found when trying to get lock; try
restarting transaction]
[to_string] => [db_error: message="DB Error: unknown error" code=-1 mode=callback
callback=CRM_Core_Error::handle prefix="" info="INSERT INTO civicrm_activity_contact
(activity_id , contact_id , record_type_id ) VALUES ( 1145 , 8954 , 2 ) [nativecode=1213 **
Deadlock found when trying to get lock; try restarting transaction]"]
)
Sep 17 04:52:41 [info] $backTrace = #0
/var/www/example.org/sites/all/modules/civicrm/CRM/Core/Error.php(197):
CRM_Core_Error::backtrace("backTrace", TRUE)
#1 [internal function](): CRM_Core_Error::handle(Object(DB_Error))
#2 /var/www/example.org/sites/all/modules/civicrm/packages/PEAR.php(931):
call_user_func((Array:2), Object(DB_Error))
#3 /var/www/example.org/sites/all/modules/civicrm/packages/DB.php(969): PEAR_Error>PEAR_Error("DB Error: unknown error", -1, 16, (Array:2), "INSERT INTO
civicrm_activity_contact (activity_id , contact_id , record_type_...")
#4 /var/www/example.org/sites/all/modules/civicrm/packages/PEAR.php(564): DB_Error>DB_Error(-1, 16, (Array:2), "INSERT INTO civicrm_activity_contact (activity_id , contact_id ,
record_type_...")
#5 /var/www/example.org/sites/all/modules/civicrm/packages/DB/common.php(1905): PEAR>raiseError(NULL, -1, NULL, NULL, "INSERT INTO civicrm_activity_contact (activity_id ,
contact_id , record_type_...", "DB_Error", TRUE)
#6 /var/www/example.org/sites/all/modules/civicrm/packages/DB/mysql.php(898): DB_common>raiseError(-1, NULL, NULL, NULL, "1213 ** Deadlock found when trying to get lock; try
restarting transaction")
#7 /var/www/example.org/sites/all/modules/civicrm/packages/DB/mysql.php(327): DB_mysql>mysqlRaiseError()
#8 /var/www/example.org/sites/all/modules/civicrm/packages/DB/common.php(1216):
DB_mysql->simpleQuery("INSERT INTO civicrm_activity_contact (activity_id , contact_id ,
record_type_...")
#9 /var/www/example.org/sites/all/modules/civicrm/packages/DB/DataObject.php(2421):
DB_common->query("INSERT INTO civicrm_activity_contact (activity_id , contact_id ,
record_type_...")
#10 /var/www/example.org/sites/all/modules/civicrm/packages/DB/DataObject.php(1055):
DB_DataObject->query("INSERT INTO civicrm_activity_contact (activity_id , contact_id ,
record_type...")
#11 /var/www/example.org/sites/all/modules/civicrm/CRM/Core/DAO.php(278):
DB_DataObject->insert()
#12
/var/www/example.org/sites/all/modules/civicrm/CRM/Activity/BAO/ActivityContact.php(63):
CRM_Core_DAO->save()
#13 /var/www/example.org/sites/all/modules/civicrm/CRM/Activity/BAO/Activity.php(364):
CRM_Activity_BAO_ActivityContact::create((Array:3))
#14 /var/www/example.org/sites/all/modules/civicrm/CRM/Activity/BAO/Activity.php(1859):
CRM_Activity_BAO_Activity::create((Array:10))
#15 /var/www/example.org/sites/all/modules/civicrm/CRM/Core/Payment/BaseIPN.php(586):
CRM_Activity_BAO_Activity::addActivity(Object(CRM_Contribute_BAO_Contribution),
NULL, NULL)
#16
/var/www/example.org/sites/all/modules/civicrm/CRM/Core/Payment/AuthorizeNetIPN.php(183):
CRM_Core_Payment_BaseIPN->completeTransaction((Array:28), (Array:6), (Array:8),
Object(CRM_Core_Transaction), Object(CRM_Contribute_BAO_ContributionRecur))
#17
/var/www/example.org/sites/all/modules/civicrm/CRM/Core/Payment/AuthorizeNetIPN.php(72):
CRM_Core_Payment_AuthorizeNetIPN->recur((Array:28), (Array:6), (Array:8), FALSE)
#18 /var/www/example.org/sites/all/modules/civicrm/extern/authorizeIPN.php(40):
CRM_Core_Payment_AuthorizeNetIPN->main()
#19 {main}
Sep 17 04:52:41 [info] Contribution record updated successfully
Sep 17 04:52:41 [info] Success: Database updated
Sep 17 04:52:41 [info] Contribution record updated successfully
Sep 17 04:52:41 [info] Success: Database updated
Sep 17 04:55:11 [info] Subscription payment failed - 'The credit card has expired.'
Sep 17 05:00:46 [info] Contribution record updated successfully
Sep 17 05:00:46 [info] Success: Database updated
Sep 17 05:03:25 [info] Contribution record updated successfully
Sep 17 05:03:25 [info] Success: Database updated
Sep 17 05:07:11 [info] Contribution record updated successfully
Sep 17 05:07:11 [info] Success: Database updated
--------------------------Relevant MySQL deadlock output:
Sep 17 04:52:41 mysqld: InnoDB: transactions deadlock detected, dumping detailed information.
Sep 17 04:52:41 mysqld: 140917 4:52:41
Sep 17 04:52:41 mysqld: *** (1) TRANSACTION:
Sep 17 04:52:41 mysqld: TRANSACTION 8408262, ACTIVE 0 sec inserting
Sep 17 04:52:41 mysqld: mysql tables in use 1, locked 1
Sep 17 04:52:41 mysqld: LOCK WAIT 30 lock struct(s), heap size 2496, 15 row lock(s), undo log
entries 10
Sep 17 04:52:41 mysqld: MySQL thread id 65844, OS thread handle 0xa6bceb70, query id
9658748 localhost lm update
Sep 17 04:52:41 mysqld: INSERT INTO civicrm_activity_contact (activity_id , contact_id ,
record_type_id ) VALUES ( 1143 , 8946 , 2 )
Sep 17 04:52:41 mysqld: *** (1) WAITING FOR THIS LOCK TO BE GRANTED:
Sep 17 04:52:41 mysqld: RECORD LOCKS space id 0 page no 32836 n bits 864 index
`FK_civicrm_activity_contact_activity_id` of table `lm_civi`.`civicrm_activity_contact` trx id
8408262 lock_mode X insert intention waiting
Sep 17 04:52:41 mysqld: Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact
format; info bits 0
Sep 17 04:52:41 mysqld: 0: len 8; hex 73757072656d756d; asc supremum;;
Sep 17 04:52:41 mysqld:
Sep 17 04:52:41 mysqld: *** (2) TRANSACTION:
Sep 17 04:52:41 mysqld: TRANSACTION 8408264, ACTIVE 0 sec inserting
Sep 17 04:52:41 mysqld: mysql tables in use 1, locked 1
Sep 17 04:52:41 mysqld: 30 lock struct(s), heap size 2496, 15 row lock(s), undo log entries 10
Sep 17 04:52:41 mysqld: MySQL thread id 65845, OS thread handle 0xa478bb70, query id
9658755 localhost lm update
Sep 17 04:52:41 mysqld: INSERT INTO civicrm_activity_contact (activity_id , contact_id ,
record_type_id ) VALUES ( 1144 , 8948 , 2 )
Sep 17 04:52:41 mysqld: *** (2) HOLDS THE LOCK(S):
Sep 17 04:52:41 mysqld: RECORD LOCKS space id 0 page no 32836 n bits 864 index
`FK_civicrm_activity_contact_activity_id` of table `lm_civi`.`civicrm_activity_contact` trx id
8408264 lock_mode X
Sep 17 04:52:41 mysqld: Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact
format; info bits 0
Sep 17 04:52:41 mysqld: 0: len 8; hex 73757072656d756d; asc supremum;;
Sep 17 04:52:41 mysqld:
Sep 17 04:52:41 mysqld: *** (2) WAITING FOR THIS LOCK TO BE GRANTED:
Sep 17 04:52:41 mysqld: RECORD LOCKS space id 0 page no 32836 n bits 864 index
`FK_civicrm_activity_contact_activity_id` of table `lm_civi`.`civicrm_activity_contact` trx id
8408264 lock_mode X insert intention waiting
Sep 17 04:52:41 mysqld: Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact
format; info bits 0
Sep 17 04:52:41 mysqld: 0: len 8; hex 73757072656d756d; asc supremum;;
Sep 17 04:52:41 mysqld:
Sep 17 04:52:41 mysqld: *** WE ROLL BACK TRANSACTION (2)
Sep 17 04:52:41 mysqld: InnoDB: transactions deadlock detected, dumping detailed information.
Sep 17 04:52:41 mysqld: 140917 4:52:41
Sep 17 04:52:41 mysqld: *** (1) TRANSACTION:
Sep 17 04:52:41 mysqld: TRANSACTION 8408263, ACTIVE 0 sec inserting
Sep 17 04:52:41 mysqld: mysql tables in use 1, locked 1
Sep 17 04:52:41 mysqld: LOCK WAIT 30 lock struct(s), heap size 2496, 15 row lock(s), undo log
entries 10
Sep 17 04:52:41 mysqld: MySQL thread id 65846, OS thread handle 0xa456cb70, query id
9658773 localhost lm update
Sep 17 04:52:41 mysqld: INSERT INTO civicrm_activity_contact (activity_id , contact_id ,
record_type_id ) VALUES ( 1145 , 8954 , 2 )
Sep 17 04:52:41 mysqld: *** (1) WAITING FOR THIS LOCK TO BE GRANTED:
Sep 17 04:52:41 mysqld: RECORD LOCKS space id 0 page no 32836 n bits 864 index
`FK_civicrm_activity_contact_activity_id` of table `lm_civi`.`civicrm_activity_contact` trx id
8408263 lock_mode X insert intention waiting
Sep 17 04:52:41 mysqld: Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact
format; info bits 0
Sep 17 04:52:41 mysqld: 0: len 8; hex 73757072656d756d; asc supremum;;
Sep 17 04:52:41 mysqld:
Sep 17 04:52:41 mysqld: *** (2) TRANSACTION:
Sep 17 04:52:41 mysqld: TRANSACTION 8408262, ACTIVE 0 sec inserting
Sep 17 04:52:41 mysqld: mysql tables in use 1, locked 1
Sep 17 04:52:41 mysqld: 31 lock struct(s), heap size 2496, 16 row lock(s), undo log entries 10
Sep 17 04:52:41 mysqld: MySQL thread id 65844, OS thread handle 0xa6bceb70, query id
9658748 localhost lm update
Sep 17 04:52:41 mysqld: INSERT INTO civicrm_activity_contact (activity_id , contact_id ,
record_type_id ) VALUES ( 1143 , 8946 , 2 )
Sep 17 04:52:41 mysqld: *** (2) HOLDS THE LOCK(S):
Sep 17 04:52:41 mysqld: RECORD LOCKS space id 0 page no 32836 n bits 864 index
`FK_civicrm_activity_contact_activity_id` of table `lm_civi`.`civicrm_activity_contact` trx id
8408262 lock_mode X
Sep 17 04:52:41 mysqld: Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact
format; info bits 0
Sep 17 04:52:41 mysqld: 0: len 8; hex 73757072656d756d; asc supremum;;
Sep 17 04:52:41 mysqld:
Sep 17 04:52:41 mysqld: *** (2) WAITING FOR THIS LOCK TO BE GRANTED:
Sep 17 04:52:41 mysqld: RECORD LOCKS space id 0 page no 32836 n bits 864 index
`FK_civicrm_activity_contact_activity_id` of table `lm_civi`.`civicrm_activity_contact` trx id
8408262 lock_mode X insert intention waiting
Sep 17 04:52:41 mysqld: Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact
format; info bits 0
Sep 17 04:52:41 mysqld: 0: len 8; hex 73757072656d756d; asc supremum;;
Sep 17 04:52:41 mysqld:
Sep 17 04:52:41 mysqld: *** WE ROLL BACK TRANSACTION (1)
--------------------------The curious part about this is that it's deadlocking on a foreign key constraint when trying to create
the contacts of the activity for the contribution, not than the contribution itself, which I would
assume procedurally has already created. I've looked through the code a bit and I'm pretty sure this
is the relevant section, CRM/Core/Payment/BaseIPN.php, line 578 in 4.4.7 branch.
// create an activity record
if ($input['component'] == 'contribute') {
//
$targetContactID = NULL;
if (!empty($ids['related_contact']))
{ $targetContactID = $contribution->contact_id; $contribution->contact_id =
$ids['related_contact']; }
CRM_Activity_BAO_Activity::addActivity($contribution, NULL, $targetContactID);
// event
}
else
{ CRM_Activity_BAO_Activity::addActivity($participant); }
CRM_Core_Error::debug_log_message("Contribution record updated successfully");
But we're not sure how to fix the problem. It doesn't seem like a MySQL server configuration fix
(as far as I've read). We're (Palante) willing to fix this problem but wanted to get some guidance
on the best approach to take.
Comments
Comment by Michael Lueck [ 07/Jan/15 ]
We have one client who has encountered this same bug. I opened a forum report about it here:
"Recurring contribution failure error 'Deadlock found...' "
http://forum.civicrm.org/index.php/topic,35249.msg149829.html
What further details could I provide to assist with getting to the bottom of this error?
I am thankful,
Michael
Comment by David Knoll [ 21/Jan/15 ]
We've got something like this too:
http://forum.civicrm.org/index.php/topic,35409.0.html
Comment by Tim Otten [ 23/Jan/15 ]
A few general questions (not based on any real understanding the particular instance of the
particular problem):




The reports so far sound like they've come from organic circumstances in production
environments. Has anyone tried (successfully or unsuccessfully) to reproduce the
problem in a controlled environment (developer/non-production)? With what steps?
Does anyone have logs about cron tasks or background processing that might be
concurrent with the deadlocks?
=> Significance: The archtypal deadlock is a circular wait where two concurrent
processes try to lock the same resources in different orders. This is more likely to happen
when the two processes follow different logic – which suggests that some other
(qualitatively different) process might be involved.
Do we know any more about the circumstance in which Authorize.net sends back
multiple notifications? For example, are all the notifications related to the same contact
and the same contribution – or to different contacts and different contributions? The
example log above seems to show different contacts+activities – I'm just
verbalizing/asking to verify.
Reportedly, the keys+constraints can impact locking+deadlocking behavior. It would be
good verify that everyone has the same schema. Run "show create table
civicrm_activity_contact" and "show create table civicrm_activity." This is what I see on
a clean installation: https://gist.github.com/totten/e838889383e34db8cad6
Comment by Tim Otten [ 23/Jan/15 ]
OK, I found a way to reproduce.
1. Copy the script https://gist.github.com/totten/7fff23e91d8fc777fec6 to /tmp/randact.php
2. Open 2 or 3 terminals
3. In each terminal, run "drush scr /tmp/randact.php"
Comment by Tim Otten [ 23/Jan/15 ]
Let me rephrase that. I found a way to produce a deadlock on civicrm_activity_contact. It's not
really reasonable that this is a deadlock – each process is (by definition) dealing with a different
set of activities and contacts. There's something wrong with the grain of the lock.
This is not necessarily the same deadlock observed in processing recurring contributions or
SMSs, but all three involve deadlocking on the same table (civicrm_activity_contact), and (in
the simple/easily reproduced case) it seems like unreasonable behavior.
Comment by Michael Lueck [ 23/Jan/15 ]
In our case, the only cron tasks enabled on the domain are cleanup and process_mailing. I do not
believe this client is making use of CiviMail, so I will disable that job once confirmed.
Cron is run via Drush on the hour. The crash I reported was at 08 of the hour, so definitely cron
would not still be running.
Comment by David Knoll [ 23/Jan/15 ]
Tim: I compared the table schema on my local dev install (which comes from a mysqldump of
production). civicrm_activity was almost the same except two FKs appeared in a different order.
My civicrm_activity_contact has an additional KEY line:
KEY `FK_civicrm_activity_contact_activity_id` (`activity_id`),
That script run on my local system, the first time with 2 copies didn't deadlock. The second time
with 3 copies 2 finished successfully and 1 deadlocked on civicrm_acl_cache. The third time
with 3 copies didn't deadlock.
Comment by Joseph Lacey [ 26/Jan/15 ]
> * The reports so far sound like they've come from organic circumstances in production
environments. Has anyone tried (successfully or unsuccessfully) to reproduce the problem in a
controlled environment (developer/non-production)? With what steps?
We haven't tried to reproduce it. We found the deadlock errors and wanted to report them to get
some feedback from the community about next steps and whether others had similar issues.
> * Does anyone have logs about cron tasks or background processing that might be concurrent
with the deadlocks?
Our production server (Debian) is a fairly standard web server. Runs the LAMP stack, Postfix
and Varnish along with Debian-specific processes. It only keeps syslog output for a month, so
I'll need to regenerate the CiviCRM log and MySQL deadlock log output along with this to give
the overall picture. Shall I post it here?
> * Do we know any more about the circumstance in which Authorize.net sends back multiple
notifications? For example, are all the notifications related to the same contact and the same
contribution – or to different contacts and different contributions? The example log above seems
to show different contacts+activities – I'm just verbalizing/asking to verify.
No, it seems that in Palante's case, it's a random pair out of the 11 each time.
> * Reportedly, the keys+constraints can impact locking+deadlocking behavior. It would be
good verify that everyone has the same schema. Run "show create table
civicrm_activity_contact" and "show create table civicrm_activity." This is what I see on a clean
installation: https://gist.github.com/totten/e838889383e34db8cad6
Our case is 4.4.x. Would these have changed?
Comment by Kurund Jalmi [ 28/Jan/15 ]
We were finally able to replicate this and we think the issue might be with delete operation
blocking inserts when there are too many simultaneous requests. We will update here with the
PR.
Comment by pratiksha [ 29/Jan/15 ]
Submitted PR https://github.com/civicrm/civicrm-core/pull/5034
Comment by Joseph Lacey [ 29/Jan/15 ]
This is great. Thanks, y'all. I've patched the instance in question. If you want to assign this ticket
to me, I'll report back after the next round of recurring contribution ping backs to make sure this
resolved our issue.
Comment by Kurund Jalmi [ 30/Jan/15 ]
Joseph, here is the updated PR: https://github.com/civicrm/civicrm-core/pull/5039 ( fixed the
regression reported by test )
Comment by Kurund Jalmi [ 05/Feb/15 ]
Joseph,
Did you see any deadlocks after applying https://github.com/civicrm/civicrm-core/pull/5039
Comment by Joseph Lacey [ 05/Feb/15 ]
Hey Kurund,
Our current problem case are recurring transactions that happen on the 17th of each month. I'll
post a reply when those come through about whether this is fixed or not.
Joseph
Comment by Kurund Jalmi [ 05/Feb/15 ]
Great, thank you ( just to confirm: I hope you are using latest PR:
https://github.com/civicrm/civicrm-core/pull/5039, initial one had test regressions.)
Comment by Joseph Lacey [ 05/Feb/15 ]
Ah, thanks for the heads up, Kurund. I'll let you know what we find.
Comment by Joseph Lacey [ 18/Feb/15 ]
All the recurring contributions from Authorize.net completed without error yesterday.
Joseph
Comment by David Greenberg [ 18/Feb/15 ]
Based on last comment from Joseph, looks like this is resolved. We can reopen if further issues
pop up.
Comment by Eileen McNaughton [ 11/Oct/15 ]
We are now seeing similar results from the insert into civicrm_line_item table for Authorize.net
- at least on memberships
Generated at Sun Mar 06 09:43:23 UTC 2016 using JIRA 6.2#6252sha1:aa343257d4ce030d9cb8c531be520be9fac1c996.