Posted by nattyweb on Wed, Jun 29, 2022 - 7:05am

I've posted this on the Github feed for the Feeds module - adding here in case there's a different audience...

I used Feeds to import users on a D7 site many times in the past, so am confused why this isn't working in Backdrop.
I have an upgraded site with hundreds of users successfully carried over from a D7 site. When I try to use Feeds to import and update changed user accounts, I'm getting this error:

Integrity constraint violation: 1062 Duplicate entry '80-5' for key 'PRIMARY'

(where 80 is the id of an existing user).

I've set the User Processor settings to either of "Insert new user accounts" or "Do not insert new user accounts" (makes no difference eitehr way) and "Update existing user accounts" is on.

and the mapping such that user is a unique identifier.

Any clues about what's going wrong here? I'm not sure where to look to find more detail.

Recent log messages shows:
PDOException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '80-5' for key 'PRIMARY': INSERT INTO {users_roles} (uid, role) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1), ...

Thanks
Martin

 

Most helpful answers

Hi Martin

Hi Martin

I'm wondering if Feeds Tamper could be your friend here? 

https://github.com/backdrop-contrib/feeds_tamper

Create the roles from scratch in backdrop so they have a machine name

Setup your feed with the tamper to map the old drupal machine number to the new backdrop machine name then perhaps delete and re-import (although update might work).

Martin

Comments

Wed, Jun 29, 2022 - 12:05pm by yorkshirepudding

It looks like this is happening outside the feeds module so in the DB write itself

It's not the user per se that is the duplicate but the combination of user id and role id in the users_roles table (both fields are Primary).

Have the role Ids changed in backdrop from drupal?  Look on admin/config/people/roles (and equivalent in Drupal) and compare.

how are you generating the feed on the Drupal side? Is it possible that the join of users and roles is creating duplicate combinations?

Wed, Jun 29, 2022 - 3:07pm by argiepiano

A couple of other things to keep in mind:

- Roles in Backdrop do not use numeric IDs, but machine names. I see that the database insert command is trying to insert a numeric id as a role

- Is this a site that was upgraded from D7, or created from scratch in Backdrop? If created by scratch, then the user roles will be identified machine names. Often D7 sites upgraded to Backdrop keep the numeric ID for roles, but this can create problems in the long run. Check your files/confix_NNN folder and look for files that start with user.role. Do you see "user.role.administrator.json" for example? Or do you see "user.role.5.json"

- Thirdly, if this is an upgraded D7 site, you may have already imported the users_roles table but not the users table. So, the users_roles may be stale and contain entries for the old site. This would create issues when you save new users who have a specific role who happen to have the same user ID of a stale entry in the table. 

Thu, Jun 30, 2022 - 3:22am by nattyweb

Thanks for the replies - clearly on the right track toward helping me!

To answer both sets of questions:

  • The site is an upgraded D7 site.
  • The role names transferred in the upgrade are as on the D7 site, but the machine names are numeric.
  • In the config dir I have for example "user.role.5.json".
  • both users and users_roles tables were successfully imported during the upgrade.
  • The export from the D7 site includes the username as well as the user role, but the role is exported as the name of the role (eg 'member').

I'm not exactly clear on what I need to do next, short of removing all users and roles and recreating them. Any ideas?

Thu, Jun 30, 2022 - 7:30am by argiepiano

nattywebdev:

Without debugging it's difficult to tell if the problem is Feeds or is created by core upon saving to the upgraded users_roles table.

So, maybe a bit more intuitive testing is needed before doing a deep debug. I'd try the following:

1. Start by upgrading your site, and then EMPTYING the users_roles table completely from the upgraded Backdrop site, then try the Feeds import again. Since the problem is overwriting records in the users_roles table, by emptying it, your Feeds process should be able to write those. The risk is that this error maybe happening for SOME users and not others, in which case you may end up with some users with no roles in the table. Check around for that

2. A bit more risky - try emptying both the users and the users_roles tables and then do the Feeds import. Not sure if your users create content. If that's the case, then this can create problems, as the imported users will probably end up with a uid different from the one in your D7 site, and therefore the nodes' uid may point at a user different from the one who actually created that node

I suspect that the problem is not with Feeds, but with a combination of it with numeric roles and core's user update hooks. If the above doesn't work, then we'll need to dig deeper. 

Thu, Jun 30, 2022 - 7:28am by argiepiano

I edited my response a bit - please read it in the forum, not in the email notification.

A more pointed, narrower approach is to simply open the users_roles table and only remove the record that's producing the problem, in this case uid = 80 and role = 5 

Thu, Jun 30, 2022 - 3:38am by yorkshirepudding

Hi Martin

I'm wondering if Feeds Tamper could be your friend here? 

https://github.com/backdrop-contrib/feeds_tamper

Create the roles from scratch in backdrop so they have a machine name

Setup your feed with the tamper to map the old drupal machine number to the new backdrop machine name then perhaps delete and re-import (although update might work).

Martin

Thu, Jun 30, 2022 - 7:35am by nattyweb

Thank you for your advice @yorkshirepudding and @argiepiano - I really appreciate it.

I think it is now resolved after I removed the roles and recreated them on the Backdrop site. Once I'd done that, the import (of a sample) of users worked without errors. There are over 800 users so I'm not going to test it with all until I've ironed out all other issues with user importing - mostly look straightforward (now having fun with addresses from other countries).

@yorkshirepudding - I was already using Feeds Tamper BUT I hadn't accounted for users with multiple roles, so adding explode to the actions nailed that.

 

Thu, Jun 30, 2022 - 8:37am by yorkshirepudding

@nattywebdev - I'm glad we were able to help. 

PS - I'm UK based too