Storing projects - the idea

How to store a project to preserve it for the future ?
My project have up to three components.
A description/documentation. some code, schematics.

I like the idea to have my work not only stored in different servers like Bitbucket or EasyEDA or Blogger but also stored on my server, locally.



Here my idea about how to store projects.

Typically the blog contains one or more articles about the project, description/documentation, etc.
Bitbucket hosts software/documentation/etc.
If schematics are based on Eagle CAD then also that will be in Bitbucket.
EasyEDA schematics and PCB are available only on EasyEDA website.

The final goal is to have an automatic way to :

• collect all these documents every time they are modified
• save them locally in a server
• backup them on a long storage

This project will use Jenkins and Drobo to store locally my work while keeping it on external servers as well.

The idea

The idea is to have Jenkins monitoring activity somehow on the different servers and then copy locally when modifications occurs.
Bitbucket should have a way to inform (trigger) Jenkins when a modification occurs, EasyEDA and the Blog will be polled by Jenkins, maybe once a week.
When modifications occurs Jenkins will also save the projects on Drobo.

Bitbucket

Jenkins pipeline allows to download a repository automatically.
A trigger can be set so that Jenkins can access bitbucket in order to pull a repository when the repository is modified or just polling Bitbucket every week.

EasyEDA

From the UI is quite easy to download the entire project.
Going into Manage Project and selecting Advanced in Settings, a menu appears where you can download the entire project as zip file.
The zip file contains the latest json files for schematics and PCBs.
However the goal here is to have this done by Jenkins, not via web.

The main problem seems to login in on unmanned way, no API seems available.
The download button in the web appears only if you are logged in and the link for the download is created every time.

Until EasyEDA doesn't prepare a way to automatically download  a project, the only way to backup is to manually download  the zip file somewhere, where the Jenkins can pick up the zip file and save it.

For now this the procedure :

• manually download a project from the EasyEda website
• copy the zip file on a specific area in Dropbox
• let Jenkins periodically check in the Dropbox area for newer versions of the project

Jenkins can then check if the Dropbox area contains a file.
If it exist, the file is copied in the project area and parsed and older versions of the project in Dropbox are eliminated.
Only the latest version of the project is kept in Dropbox as well.

The downloaded zip file name is set up in this way :

Project_<project name>_<date + time>.zip

• Project
  Preamble - fixed
• project name
  The name of the project as set in EasyEDA
• date time
  A single string with : yyyyMMddHHmmss

Example : Project_WellBatt_20190109151443.zip

Knowing this data is possible to update the metadata of the latest project file uploaded.

Blog

Probably the best way is to convert the blog in PDF and save the pdf file locally.
PrintFrendly has an API service that allows to convert and download a blog entry in PDF, but it cost 20$ a month of subscription.
Another possible solution is to install a utility on the server to do so.

MetaData

Could be useful to store the metadata for a project in a database to facilitate to manage different versions/location/retrieval.
Considering to involve also Artifactory  ?

Final storing

Each project will be saved in Drobo.
So under Drobo exists a main directory where all the projects are stored.
Depending what information are stored in the directory is possible to have three files :

• a PDF file with the latest Blog entry/entries for the project
• a zip file containing the EasyEDA schematics/PCB
• a tar.gz file from Bitbucket

Future articles will describe more in detail some components of this project.