Ticket #91 (assigned enhancement)

Opened 4 years ago

Last modified 2 years ago

export to XML or RDF

Reported by: kees.vanbochove@… Owned by: s.h.sikkema@…
Priority: major Milestone: Must have
Component: Export functionality Version:
Keywords: Cc: robert@…
Product: Operating system:
URL: Hardware:

Description

It would be very nice to get an output of a study in XML or RDF form, to improve exchange-ability with other tools and platforms.

Attachments

export.xml Download (110.1 KB) - added by business@… 3 years ago.
First version of XML export of a study

Change History

Changed 3 years ago by business@…

  • status changed from new to assigned
  • owner set to jahn.saito@…
  • milestone set to 0.8

Changed 3 years ago by business@…

Have in mind that this data should be able to be imported to another running GSCF instance with possibly other database identifiers and connected modules.

Changed 3 years ago by business@…

It would be wise to also export the templates used in the study along with the study (#76)

Changed 3 years ago by business@…

First version of XML export of a study

Changed 3 years ago by business@…

  • cc robert@… added

Received an example XML of Jahn today, attaching it to the ticket... will provide feedback on it soon.
See

Error: Failed to load processor File
No macro or processor named 'File' found


Changed 3 years ago by business@…

OK, I reviewed the example XML in detail. It looks good! I have the following comments:

  • I suppose the id numbering is for internal XML reference purposes only, and that the importer will generate new ids from the database. I guess that's your aim but just to be clear.
  • Templates are currently not exported, however, you do export template fields and you also refer to templates in many entities (which extend TemplateEntity?, so you have to). So my guess is that we also need to export the templates. However, you could re-use some of the code that Robert wrote for template XML import to import the templates. My guess would be that the study XML importer should carry out the following steps:
    • load all templates XML first
    • for each template, determine if there is already exactly the same template in the database (use Robert's code!), if so, store the reference to that template, otherwise, import the apparently new template
    • after that, import the studies one by one (you have to build up the study entities in memory, maybe make maps that can translate the current id's in the XML to the corresponding memory objects? Also you need to be careful of the sequence of entity types, they depend on each other. I guess the order should be something like: Subjects, EventGroups?, Events, SamplingEvents?, Samples, Assays, Study and related. And no doubt you would have to do many addTo... assignments in a second round to update the bi-directional hasMany/belongsTo relations. Look to BootstrapStudies? for examples.
  • For Persons/StudyPersons?, Publications, and the owner/readers/writers SecUser? fields, you will have to follow the same pattern as with Template, as these are also database-wide entries (re-uses across multiple studies). So first add them to the database if they are not already there, and then import the studies. Going into details:
    • Persons: they can be added if they don't exist already based on name and affiliations. Otherwise use the existing db record.
    • SecUser? fields: I think we should enforce here that the user(s) already exist in the database, otherwise we have to transfer passwords etc., we don't want to go into that. Which means users would be the only type that we would have to rebuild manually between db export/imports, but I can live with that as this will be relegated to a third party ID server anyway in the future.
    • Publications: just check if the Pubmed ID is already in there, if so, use the record, otherwise add it (based on the XML properties so you don't have to invoke webservices)

Changed 3 years ago by business@…

  • milestone changed from Could have to 0.6.6

Changed 3 years ago by work@…

  • milestone changed from 0.6.6 to 0.7

Changed 3 years ago by work@…

  • priority changed from major to blocker

Changed 3 years ago by work@…

Jahn, what is the status? I have made this a blocker now....

Changed 3 years ago by work@…

  • milestone changed from 0.7.0 to 0.7.1

no feedback... moving to 0.7.1

Changed 3 years ago by business@…

  • owner changed from jahn.saito@… to s.h.sikkema@…

Changed 3 years ago by work@…

  • milestone changed from 0.7.1 to 0.8.1

Changed 3 years ago by robert@…

It is important that the studyToken, sampleToken and assayToken are exported as well, and also correctly imported. I've worked on an exporter of sequencing module data, and I have to be able to connect the data to the correct study/assay/sample again.

Changed 3 years ago by business@…

  • milestone changed from 0.8.1 to 0.8.2

Changed 3 years ago by business@…

  • milestone changed from 0.8.2 to Must have

Changed 2 years ago by business@…

  • priority changed from blocker to major
Note: See TracTickets for help on using tickets.