Skip to main content

From September 30, our extraction system for FTP and SFTP sources will be updated to ensure better consistency between your published datasets and their original source.

🔎 What’s changing

Currently, extracted files are cached even if they are deleted from the source. With the new process, only files that are present in your external source at the time of republishing will be kept. Deleted files will no longer be retained in Opendatasoft.

👉 Actions to take before September 30

If there are deleted files still stored in your cache, you can:

  • Keep them by adding them back to your source, or by exporting your current dataset and reintegrating it into the source.
  • Permanently delete them by clearing the cache for the relevant datasets, starting from a clean slate.

 

 

 

 

 

 

 

 

 

🚀 Benefits of this update

  • Faster and smoother loading in the back office
  • Data is always aligned with your external source
  • Improved traceability of extraction errors

💡 Need help?

Contact your Customer Success Manager, our Support team. 

Hello.

Will this new process increase the automatic reindexation time ? ( the cache system will still exist ? )

And , does it mean from 30 September, dataset records whom source file is deleted from sftp will be ‘deleted’ from dataset on next scheduled index update ?
 

Or will they still persist in the dataset until a complete depublish/publish ( that is the current way it work if i’m not wrong )


 

Hello, 

Thank you for your interest in this new process.

  • no, the new process does not increase the automatic re-indexation time. The cache system will still exist, only it is the source that will be used to list the files that will be processed. This will ensure a consistency between published datasets and their source.
  • yes indeed, it means that from 30 September, dataset records whom source file is deleted from sftp will be ‘deleted’ from dataset on next scheduled index update. As suggested in the above post, if these records need to remain in the dataset, you can add the deleted files back to your source if you still have them, or export your current dataset and reintegrate this export in your source.
Terms & ConditionsCookie settings
Opendatasoft Terms & Conditions Cookie settings