I think I found an explanation for a number of issues that have been reported... I started having a lot of problems with the synchronization such as: the app showing that contacts were deleted that were not, contacts being duplicated, performance being slow, etc.
I had responded on one of the tickets that I noticed that my contacts had grown to over 1024, which was a suspicious number (to those with a developer background). It turns out that I think the actual "magic number" where trouble can happen is actually 1000, because I found that Google limits the number of contacts that can be created by a user to 1000 per day: look for "contacts created" on this page: https://developers.google.com/apps-script/guides/services/quotas. (Google likely created that limit thinking that a person should be able to reasonably be able to add 1000 new contacts a day, not realizing that 1000 or even a few 1000 would be possible via a program like GO Contact Sync Mod.
I've seen some things that can cause a re-synchronization of all contacts - which is possibly a bug itself, but when that happens, if the number of contacts re-created on Google exceed 1000, I think Google may be preventing more from being created. I'm not sure if there is a return value that is passed back for those circumstances, but whatever is happening, it is confusing the GO Contact Sync Mod app. At one point, my contacts were so messed up that I went through all my Outlook contacts I did a "Reset matches", and tried to start fresh, but I noticed, again, that about as many contacts as exceeded 1000 would still have problems. I suspect that some performance issues mentioned may also be tied to this as when the Google limit is hit, something abnormal (and maybe time-consuming) probably starts happening.
In my case, I went through my contacts again and reduced them to being less than 1000 and I haven't seen the issues again. Still, there may be a way for the application to handle the >1000 limit more gracefully and/or less confusingly.
The link I had provided has a period after it that may cause the link to not work. Here it is without the period: https://developers.google.com/apps-script/guides/services/quotas
I have reduced the pagesize to 250 when loading appointments and contacts. Maybe this fixed this issue, at least then it should not retrieve more than 1000 entries in one go. Can you please retry with new version 4.1.32 (will be released today)
I should have replied here - dup msg as I'm not sure I can delete the other -
Florian - Can you explain the 250 page size. For example if I want to block lunch time for the year, would that fail - it being 260 weekdays?
Hi, I forgot to mention, I did the pageSize to 256, because this was working in old Google Contact API. Nevertheless, don't worry about it, it will not restrict the time range you use, it doesn't have to do anything with weekdays, it just sets the number of Google contacts retrieved with one request, similar to a WebPage, where you see only the first 256 entries on page one, and if you click on second page, you will get the next 256 entries, until the end of the entries is reached. GCSM loops thru all pages and gets all contacts and appointments, just the number of contacts/appointments retrieved in one request is now limited to 256 to avoid swallowed entries or page overlap issues (even if we would not set a page size, there will be a default page size, and this one seems to have a problem, therefore I switched back to the successfully used page size 256 in old GoogleContact API).
Hi Florian. Thanks for the response. My understanding from the Google "quotas" (limits) page I had linked is that there seems to be a limit of 1000 "Contacts created" per day, so I'm not optimistic that reducing the number of items loaded/retrieved will help the overall issue. I think you would need to reduce the number of Google contacts created within a day to <1000. That seems possible based on the dates that are being stored with the process, but it would warrant a message on the "Sync Details / Log", though, so that the user understands why a particular sync may not be complete until another subsequent sync on a later day.
Simply reducing the number of retrievals would possibly cause more issues than it solves. I would be cautious of implementing that release live for everyone.
The ideal thing would be if someone had an inside contact with Google to tell them about the issue to see if the limit could be increased to something more reasonable.
Last edit: Fen Frehner 2022-08-06
Hi, I think setting the pageSize manually doesn't have any risk, because also if not set, there is a default pageSize. In the past (before switching from old GoogleContact API to GooglePeople API, we had a fetch size of 256 already and it worked much better than today, we never had this kind of issues with the old GoogleContact API. But it is like it is, Google ended support for GoogleContact API, and we have to live with it and fix/workaround the issues we inherit now. I tested it also on my side with ~600 contacts and at least it worked fine with pageSize 256. But you are right, it will not fix the limit of 1000 contacts to be created. Nevertheless it might fix some overlap issue if there is a problem with fetching the 999th and then the 1000th and then the 1001st contact. maybe there is a bug, because we have these "deleted" issue not only for recreating all contacts, but also for updating. I think there is a Google bug, which somehow doesn't retrieve the one or other contact from Google, and this causes GCSM to think, it was deleted. Therefore I am quite confident, playing around with the page size (choosing a reasonable size) would at least improve the situation. We did already a lot of fixes/workaround to avoid that GCSM thinks, a contact was deleted in new People API, e.g. trying to reload the assumed deleted contact again with several retries, which could cause the performance issues reported, because if this happens frequently (e.g. for each contact between 1000 and 2000), the performance would go down drastically.
In a nutshell: I don't think, the problem comes from creating more than 1000 contacts, I belief it has to do either with the retrieval process (overlapping of fetch pages) or with the hashTag we use, if the id property exceeds the allowed size (e.g. the Google account name is too big to save into Google Property, then we hash it to make a unique, but short id. This hash is different between x86 and 64bit, and beause we recently switched from x86 to 64bit to avoid Memory problems, all ids generated with the x86 version don't match aynmore with the hash we compare in the 64bit. In this case, I always recommend to reset the matches and resync. If you here choose to delete the Google contacts, it will start to recreate all the contacts, and of course, if there are more than 1000, the Google limit will trigger. Here I would then try to NOT delete the Google contacts when asked, because GCSM also tries to match existing contacts by unique properties (e.g. Email or Mobile Phone).
Nevertheless, I think the limit of 1000 creates is annoying, but shouldn't be a problem for GCSM, because if it created already 1000 contacts and fails to create the 1001st, it should show/log an exception and skip the contact (not creating a match yet). The next day when it runs again, it should then create the next 1000 contacts, and so on, until all x000 contacts have been created and matched. From that time on, it should just update, and this should not have any limit. If this is not working like that, e.g. GCSM creates a match but later not finds the match (because not found on Google side) and as a consequence thinks the Google contact was deleted and deletes the Outlook contact, then we should fix this instead, but looking into a code, the match should be created, if Inserting Google Contact fails, see below code. If this is not done, we might have to review all scenarios, if we do this properly in GCSM. Can you try again with latest 4.1.32 version and if still not working, provide us the detailled log?
private void SaveOutlookContact(ref Person gc, ContactItem oc)
{
ContactPropertiesUtils.SetOutlookGoogleId(oc, gc);
Save(ref oc);
ContactPropertiesUtils.SetGoogleOutlookId(gc, oc);
var gc1 = SaveGoogleContact(gc);
if (gc1 != null)
{
gc = gc1;
ContactPropertiesUtils.SetOutlookGoogleId(oc, gc);
Save(ref oc);
if (Synchronizer.SyncPhotos)
{
SaveOutlookPhoto(gc, oc);
}
}
else
{
ContactPropertiesUtils.ResetOutlookGoogleId(this, oc);
Save(ref oc);
}
}
See also Grigios comment in https://sourceforge.net/p/googlesyncmod/bugs/1324: "seems to always happen on the "last + 1" contact.
In my case, I have 564 contacts, and this error appears when GCSM tries to synchronise the "565th contact". It seems to me that GCSM creates this "phantom contact" for some cause...In my case, I have 564 contacts, and this error appears when GCSM tries to synchronise the "565th contact". It seems to me that GCSM creates this "phantom contact" for some cause..."
Last edit: Florian Saller 2022-08-07
Thanks Florian, this is all good to know.
Hash ID 32 vs. 64 bit. How would one know that they needed to reset the IDs to the 64bit hash?
And now I think I understand why when I sync my laptop that it takes so long. In my case I have one main desktop and two laptops (small and large screen) and before traveling I sync contacts and appts. This sync take a much longer time as does the next main desktop sync. I assume it is dealing with ID changes/hashs
Is there a way I can make a change on my end that all three can share the same hash table and sync at the same speed? Not asking for a new feature as I can just get coffee while it is working :-)
Cheers!
Hi, I don't know any method to detect if it is Hash 32 vs. 64 bit. It is just a different hash in both systems. Sometimes even when upgrading to new .NET Framework version, the hash method can change, so we should avoid upgrading as long as possible.
The only method how to recover is to resetMatches once (on each Laptop/PC) to generate new Hash codes with latest 64bit version. You can try without deleting any contacts on any side, because GCSM will try to rematch with unique properties, but this could cause also a (onetime) very long run, because matching by properties takes much longer than by IDs. After matching, the system has created new Hash codes and from that time on the match will be done by ID.
Fen - Last time (couple years back) I ran into the issue of having to resync (Google to Outlook) with Outlook being my master. My solution was able to delete all my Google contacts, then in Outlook export CSV and import (over 1,000 contacts) into Google Contacts using their import.
Did you happen to try that and it not work?
Hi Chess. I was going through e-mail and saw that I didn't respond to your question. Sorry about that. I'm familiar with the Google contacts import but I've used it on a limited basis only - like to share a subset of my contacts with a family member. I do not know if the Import feature has the same limitation, and at this point, I've been keeping my contacts just below 1000 to avoid the stated limitation; but I'll keep the Import in mind to try if it makes sense later. Thanks.
Fen - FYI - I upgraded my two laptops (Win 10) with new devices and let them sync contacts & appts against Google - no issues with 1,726 contacts syncing to them. I'm currently on 4.1.33, but was probably on 4.1.32 when I did the upgrade. This may be different from your OP as the contact sync was from Google to the laptops (no contacts added to Google). But no issues keeping them both and my desktop in sync with Google contacts (and appts). And I know I had exported from Outlook and imported to Google before I found GO Sync MOD, so I may have never exceeded 1,000 to Google at any time. And maybe importing file within Google contacts may not have a 1,000 limit.
Note: I still have a long sync time when I sync with a different laptop as I don't seem to be able to get the hash tables the same on all three Win 10 devices. Syncing with the same device as last sync'd is very fast.
Deleted Dup msg
Last edit: Chess McKinney 2022-08-07
Been trying to sync about ~2600 contacts on a fresh install and it's been going for several days. Any suggestions for how to accelerate based on info gleaned above? Each contact takes at least half a minute:
One update: I tried clearing all my Google Contacts and starting a fresh sync that way. It went pretty quickly for the first 1531 names, and then started doing the "magic reason" retries again and slowing down to 30-60 seconds per name from that point forward. So maybe it is some kind of upload/sync limit, but is there a workaround to avoid that?
Hi Adam. As I had mentioned, I ended up just cleaning up my contacts so that they were less then 1000. (I also created single contacts with numerous contacts put in the notes for some groups of contacts that I don't use often). There was some earlier discussion about doing a one-time Outlook export and using the Google import feature, and I think that would likely help.
That's actually what I did first, but when I did that, the sync was extremely slow. So I deleted everything from Google and started the sync with nothing on the Google side, which went faster for the first ~1500 entries or so. Is the idea that there is something like a ~1000 daily limit, so if there were some way to cap gosyncmod by day, we could avoid the problem?
Deleting everything from Google would probably not reset any counts for the day, and I'm assuming you may have tried some things (including the sync) prior to clearing the Google contacts, so conclusions should keep that limit in mind. You may want to try a day (or even 24 hours) later. I also ended up setting "Outlook to Google only" since I had mostly made updates in Outlook and the 2-way sync seemed to have complications sometimes.
I tried again today and started getting "not found for some magic reason" from contact 1. So maybe it doesn't operate by calendar day so I'll give it a couple of days with no activity and see if it works any better.
Same problem now five days later. It would take at least a couple of days running 24/7 to complete a sync at this rate.
Last edit: Adam Kessel 2023-10-25
Adam - my experience - With Google contacts deleted, then exported from Outlook (CVS) and imported to Google, the first sync did take a very long time - matching properties and creating the hash table - but after that, additional syncs are fast. Like 2 mins for my 1743 contacts.
Chess, did you notice when that first sync took a long time, was it giving the "not found for some magic reason" in the log for every single contact and trying five times for each?
It did not.
So I'm thinking there is something different going on in my case. I'm not convinced that even after it finishes this first pass over a few days it will be faster, since every entry is "not found for some magic reason" now. Each one says:
Adam, I'll just repeat that I had enough issues and strange things happening that I ended up reducing my contacts to under 1000, and I haven't had issues since that time. It's possible that there are some untrapped errors that happen because of the limit that set other things up for failure.