Ignored error from execute request leads to inserting duplicated objects

[jcavar]

Hi,
In EKCoreDataImporter there is
- (NSMutableDictionary *)fetchExistingObjectsForMapping:(EKManagedObjectMapping *)mapping
which executes fetch request.

We ran into issue when using this with NSSQLiteStoreType. The specific issue we have is when lookupValues is bigger then 999 which is max variable count for SQLite database.
See: https://github.com/mackyle/sqlite/blob/cec875a327f8d1924c60ccb20cc96d17c1868d33/src/sqliteLimit.h#L136

In this case error happens and empty array is returned.
Error Domain=NSSQLiteErrorDomain Code=1 "(null)" UserInfo={EncryptedStoreErrorMessage=too many SQL variables}
However, logic continues as there is nothing in database and inserts objects that may already be in database.
I think there are 2 ways to solve this problem. Either executing multiple requests with limit of 999 or fetching all objects and filtering them in memory.

Additionally, error should should be at least logged but probably exposed to caller as well.

[jcavar]

In memory solution could look something like this:

- (NSMutableDictionary *)fetchExistingObjectsForMapping:(EKManagedObjectMapping *)mapping
{
    NSSet * lookupValues = self.existingEntitiesPrimaryKeys[mapping.entityName];
    if (lookupValues.count == 0) return [NSMutableDictionary dictionary];

    NSFetchRequest * fetchRequest = [NSFetchRequest fetchRequestWithEntityName:mapping.entityName];
    NSPredicate * predicate = [NSPredicate predicateWithFormat:@"%K IN %@", mapping.primaryKey, lookupValues];
    //[fetchRequest setPredicate:predicate];
    [fetchRequest setFetchLimit:lookupValues.count];

    NSMutableDictionary * output = [NSMutableDictionary new];
    NSArray * existingObjects = [self.context executeFetchRequest:fetchRequest error:NULL];
    existingObjects = [existingObjects filteredArrayUsingPredicate:predicate];
    for (NSManagedObject * object in existingObjects)
    {
        output[[object valueForKey:mapping.primaryKey]] = object;
    }

    return output;
}

[jcavar]

Or maybe not even filtering objects and just adding everything to cache? This may cause memory issues but as long as objects are faults it should be fine?

[jcavar]

Hi, we filed the same issue on Yalantis/FastEasyMapping#80, and they plan to fix it. Since implementation is very similar it may solve issue here as well.

[DenTelezhkin]

This issue is stale, if you want to pursue this functionality with EasyMapping, pull request is greatly appreciated.

Closing issue for now.