Ticket #25 (closed defect: fixed)

Opened 3 years ago

Last modified 3 years ago

As of april 21st, around 16:00 about 5 code revisions have been lost

Reported by: work@… Owned by: david.vanenckevort@…
Priority: blocker Milestone:
Component: testing Version:
Keywords: Cc:

Description

This afternoon we noticed our contiuous integration build was not updating properly after code commits, and more interestingly the continuous integration build was running revision 1790 ( http://ci.nmcdsp.org) while the latest commit today was revision 1786 ( https://trac.nbic.nl/gscf/log/trunk).

As I know SVN + TRAC was migrated last week I assume that somehow a SVN state has been migrated which is lacking the last 5 commits: quote

Dear all,

Please note that the Trac server https://trac.nbic.nl/ will be
migrated to a different compute center Thursday morning 21/4. This
will result in a temporary service disruption.

More details later.

Rob

-- 
Rob W.W. Hooft
Chief Technology Officer BioAssist, Netherlands Bioinformatics Centre

From our continuous integration build log we see the continuous integration works fine until 16:00

201104211445 newer version of gscf is available
201104211445 deploying revision 1788 (version 0.8.0 / Grails 1.3.7) and undeploying revision 1787...
201104211445 uninstalling webtest plugin
201104211445 uninstalling db-util plugin
201104211445 uninstalling grom plugin
201104211445 setting build information variables in application.properties
201104211445 current workdirectory: /home/tomcat/workspace/gscf
201104211445 building new ci war...
201104211446 project deployment is still happening
201104211445 stopping gscf-0.8.0-ci
201104211445 undeploying gscf-0.8.0-ci
201104211445 deploying new ci WAR
201104211445 updating apache virtual host configuration
201104211445 reloading Apache
201104211445 done deploying gscf revision 1788 (version 0.8.0)
201104211459 newer version of gscf is available
201104211459 deploying revision 1789 (version 0.8.0 / Grails 1.3.7) and undeploying revision 1788...
201104211459 setting build information variables in application.properties
201104211459 current workdirectory: /home/tomcat/workspace/gscf
201104211459 building new ci war...
201104211500 project deployment is still happening
201104211459 stopping gscf-0.8.0-ci
201104211459 undeploying gscf-0.8.0-ci
201104211459 deploying new ci WAR
201104211459 updating apache virtual host configuration
201104211459 reloading Apache
201104211459 done deploying gscf revision 1789 (version 0.8.0)
201104211550 newer version of gscf is available
201104211550 deploying revision 1790 (version 0.8.0 / Grails 1.3.7) and undeploying revision 1789...
201104211550 uninstalling webtest plugin
201104211550 uninstalling db-util plugin
201104211550 uninstalling grom plugin
201104211550 setting build information variables in application.properties
201104211550 current workdirectory: /home/tomcat/workspace/gscf
201104211550 building new ci war...
201104211551 project deployment is still happening
201104211550 stopping gscf-0.8.0-ci
201104211550 undeploying gscf-0.8.0-ci
201104211550 deploying new ci WAR
201104211550 updating apache virtual host configuration
201104211550 reloading Apache
201104211550 done deploying gscf revision 1790 (version 0.8.0)

after which the svn state was broken:

svn: No such revision 1790
201104211619 haven't got an svn revision?
svn: No such revision 1790
svn: No such revision 1790
201104211620 haven't got an svn revision?

I don't know what happened, but we:
1. are missing 5 code revisions
2. have a conflict as code has now been committed as revision #1786 which should have been #1791

I'm sure you are aware of the seriousness of this issue, so I hope this can be resolved ASAP

Change History

Changed 3 years ago by work@…

bold same goed for nmcdsp of which the latest continuous integration build is of revision 978 while SVN is now at revision 976 (hence, 2 SVN commits are missing -->  https://trac.nbic.nl/nmcdsp/log/trunk )

201104211540 newer version of nmcdsp is available
201104211540 deploying revision 978 (version 0.1.1 / Grails 1.3.7) and undeploying revision 977...
201104211540 setting build information variables in application.properties
201104211540 current workdirectory: /home/tomcat/workspace/nmcdsp
201104211540 building new ci war...
201104211540 stopping nmcdsp-0.1.1-ci
201104211540 undeploying nmcdsp-0.1.1-ci
201104211540 deploying new ci WAR
201104211540 updating apache virtual host configuration
201104211540 reloading Apache
201104211540 done deploying nmcdsp revision 978 (version 0.1.1)
svn: No such revision 978
svn: No such revision 978
201104211620 haven't got an svn revision?

Changed 3 years ago by work@…

Our hackathon project GSCF4Molgenis is complete broken:  https://trac.nbic.nl/gscf4molgenis/

Changed 3 years ago by david.vanenckevort@…

  • status changed from new to accepted
  • owner set to david.vanenckevort@…

An update from Procolix on this matter:

They have verified the repositories as they have been transferred from loco and the commits are not present in the dump from Loco. The assumption is that the commits have been done on the old server after it was supposed to be switched to read-only.
We need to verify if the missing commits are present on the old server, and if they are reimport the repository (this will overwrite the repository as it is now).

The GSCF4Molgenis repository was missing it's Apache configuration, this has been restored manually, and the repository should be available now (same for gonl, openphacts and oauthprovider repositories).

First step now is to verify with the old server if the missing commits are there. Once we hear from Loco we will update this ticket. Procolix will try to contact Loco to get an update asap.

Changed 3 years ago by work@…

Okay clear, until this is resolved we will hold back on any commits for our projects. Perhaps it would be wise to change this SVN server to read only as well and communicate the issue directly to the users, otherwise this may lead to conflicts in other projects as well?

As for gscf and nmcdsp we do not mind if you clean the gscf / nmcdsp codebase and import the latest (accurate) dump. We have 'backupped' revision  r1786 so we can perform that change when the svn state has been resolved.

Changed 3 years ago by work@…

At 9:14 David sent the following email:

Hi everybody,
We have been investigating what went wrong and identified the problem.
We also have a list of repositories that might be affected:

conceptwiki
data-mining
dbnp-metagenomics
grails-plugins
gscf
metidb
nmcdsp
nugoctdwebapp

We still need to evaluate for each of these repositories if they are
out of sync. If that's the case we might need to do a manual merge.
This will require the assistance of the developers. If you are working
on one of the above repositories can you please hold of commits for
this morning while we investigate the issue further.

I will update you later today on the status.

With kind regards,

David van Enckevort

Changed 3 years ago by david.vanenckevort@…

The following report was made by the sysadmin team, the only remaining issue is metidb, for which I have contacted the author.

-- conceptwiki --
Migration original: Revision: 74
Loco latest: Revision: 74
Current repo: Revision: 75

geen probleem, geen actie nodig

-- data-mining --
Migration original: Revision: 255
Loco latest: Revision: 255
Current repo: Revision: 257

geen probleem, geen actie nodig

-- dbnp-metagenomics --
Migration original: Revision: 54
Loco latest: Revision: 55
Current repo: Revision: 54

Deze is in loco gecommit maar nog niet in onze repo dus nog geen 'conflict' situatie. Ik heb direct de Loco latest versie op onze repository gezet.

-- grails-plugins --
Migration original: Revision: 293
Loco latest: Revision: 293
Current repo: Revision: 293

Geen probleem, geen actie nodig

-- gscf --
Migration original: Revision: 1785
Loco latest: Revision: 1790
Current repo: Revision: 1786

Probleem, maar al afgestemd met de gebruiker dat deze 'overschreven' kan worden en dit heb ik gedaan, versie van Loco neergezet in onze repository (dus t/m 1790 maar exclusief hun commit van 1785->1786 op onze repo)

-- metidb --
Migration original: Revision: 1011
Loco latest: Revision: 1013
Current repo: Revision: 1020

Probleem, ze missen nu 2 commits 1012 en 1013. Hierover moet contact met ze worden opgenomen.

-- nmcdsp --
Migration original: Revision: 977
Loco latest: Revision: 978
Current repo: Revision: 977

Deze is in loco gecommit maar nog niet in onze repo dus nog geen 'conflict' situatie. Ik heb direct de Loco latest versie op onze repository gezet.

-- nugoctdwebapp --
Migration original: Revision: 113
Loco latest: Revision: 113
Current repo: Revision: 114

Geen probleem, geen actie nodig

Changed 3 years ago by david.vanenckevort@…

  • status changed from accepted to closed
  • resolution set to fixed

metidb changes have been manually merged.

Note: See TracTickets for help on using tickets.