Ticket #422 (closed defect: fixed)

Opened 3 years ago

Last modified 3 years ago

Error when performing action on search results on test.dbnp.org

Reported by: robert@… Owned by: work@…
Priority: major Milestone: 0.8.1
Component: General Version: 0.8.0
Keywords: Cc:
Product: Operating system:
URL: Hardware:

Description

I've sought on test.dbnp.org for all samples with study.title contains antiresdev:

 http://test.dbnp.org/advancedQuery/search?criteria.0.entityfield=Study.title&criteria.0.operator=contains&criteria.0.value=antiresdev&operator=and&entity=Sample

The search goes well, but when I select all samples and choose 'Export metadata (Mass sequencing)', a 302 error is returned. I think it has something to do with the large amounts of data. Performing the same action with only 4 samples selected works OK.

The error shown in catalina.out suggests that it has something to do with apache:

29-Apr-2011 13:52:58 org.apache.jk.common.MsgAjp cpBytes
SEVERE: Buffer overflow: buffer.len=8192 pos=42 data=23609
29-Apr-2011 13:52:58 org.apache.jk.common.MsgAjp cpBytes
SEVERE: Overflow 
java.lang.Throwable
	at org.apache.jk.common.MsgAjp.cpBytes(MsgAjp.java:196)
	at org.apache.jk.common.MsgAjp.appendByteChunk(MsgAjp.java:170)
	at org.apache.jk.common.MsgAjp.appendBytes(MsgAjp.java:155)
	at org.apache.jk.common.JkInputStream.appendHead(JkInputStream.java:324)
	at org.apache.jk.core.MsgContext.action(MsgContext.java:266)
	at org.apache.coyote.Response.action(Response.java:183)
	at org.apache.coyote.Response.sendHeaders(Response.java:379)
	at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:305)
	at org.apache.catalina.connector.OutputBuffer.close(OutputBuffer.java:273)
	at org.apache.catalina.connector.Response.finishResponse(Response.java:493)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:322)
	at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190)
	at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291)
	at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:774)
	at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
	at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:896)
	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
	at java.lang.Thread.run(Thread.java:662)

The data that is being sent (using POST) is:

Content-Type: application/x-www-form-urlencoded 
Content-Length: 4868 

actionName=excel&moduleName=Mass+Sequencing&queryId=1&id=12210&id=12219&id=12217&id=12221&id=12220&id=12213&id=12222&id=12212&id=12209&id=11439&id=12218&id=12265&id=12261&id=12280&id=12278&id=12255&id=12254&id=12234&id=12246&id=12270&id=12279&id=12235&id=12257&id=12277&id=12281&id=12264&id=12236&id=12243&id=12259&id=12240&id=12237&id=12228&id=12232&id=12266&id=11455&id=12248&id=12238&id=12267&id=12249&id=12241&id=12250&id=12256&id=11923&id=12211&id=12027&id=11931&id=12117&id=11415&id=12169&id=12292&id=12188&id=11440&id=11858&id=12139&id=11998&id=11400&id=11908&id=12079&id=12071&id=11996&id=12077&id=12294&id=12074&id=12247&id=11397&id=11936&id=12283&id=12282&id=12231&id=12286&id=12253&id=12156&id=12134&id=11918&id=12150&id=11874&id=12128&id=12148&id=12293&id=12288&id=11994&id=11387&id=12040&id=12133&id=12080&id=11993&id=12093&id=11904&id=12047&id=12010&id=12009&id=11879&id=11456&id=12072&id=12020&id=11901&id=12124&id=12200&id=11419&id=12131&id=12017&id=12127&id=12178&id=11992&id=12291&id=11911&id=11378&id=12021&id=12244&id=11408&id=12181&id=11978&id=12109&id=12180&id=12143&id=12001&id=12174&id=12120&id=12030&id=12287&id=12083&id=12276&id=12088&id=12215&id=11910&id=11905&id=11889&id=11392&id=12289&id=11399&id=12039&id=11423&id=11391&id=11454&id=12164&id=12084&id=12060&id=12214&id=12123&id=11976&id=11416&id=11434&id=12273&id=12272&id=11885&id=12242&id=11958&id=12227&id=12225&id=12230&id=11872&id=11950&id=12258&id=11949&id=11938&id=12271&id=11915&id=11384&id=12245&id=12176&id=12192&id=11916&id=11446&id=12239&id=12274&id=11862&id=12046&id=12233&id=12226&id=12285&id=12189&id=11411&id=12154&id=12202&id=12138&id=12171&id=11880&id=11977&id=12167&id=11981&id=12100&id=12223&id=11417&id=11937&id=12275&id=11899&id=12268&id=11913&id=11969&id=12229&id=11944&id=11377&id=11871&id=12190&id=11947&id=12141&id=11961&id=12073&id=12081&id=11452&id=12061&id=11429&id=12097&id=11461&id=12023&id=11413&id=12034&id=12062&id=11974&id=11865&id=12086&id=11876&id=12269&id=12216&id=12032&id=11940&id=11396&id=11964&id=12290&id=12159&id=12193&id=12262&id=12260&id=12005&id=11882&id=12069&id=12252&id=11932&id=11895&id=12224&id=11451&id=12177&id=11866&id=11951&id=12263&id=12203&id=12111&id=12036&id=11450&id=11896&id=11970&id=11875&id=12091&id=12113&id=11912&id=11924&id=11873&id=11971&id=11414&id=12201&id=11942&id=12054&id=12112&id=12208&id=12042&id=11427&id=12043&id=11894&id=11398&id=12251&id=11995&id=11863&id=12102&id=12170&id=11394&id=11892&id=11401&id=12012&id=12033&id=12085&id=11919&id=12121&id=11948&id=12104&id=12284&id=11405&id=11447&id=12115&id=11388&id=11389&id=12160&id=12149&id=12187&id=12145&id=11425&id=11403&id=12007&id=11985&id=11410&id=12110&id=12140&id=11458&id=12014&id=11997&id=12206&id=12002&id=11955&id=12015&id=11437&id=11373&id=11441&id=11442&id=11438&id=11376&id=11374&id=11375&id=12327&id=12326&id=12337&id=12338&id=12334&id=12335&id=12336&id=11877&id=11881&id=11878&id=11860&id=11897&id=11868&id=11867&id=11888&id=11898&id=11900&id=11909&id=11861&id=11869&id=11902&id=11886&id=11870&id=11864&id=11891&id=11859&id=11893&id=11907&id=11906&id=11890&id=11887&id=11857&id=11903&id=11884&id=11883&id=11963&id=11922&id=11959&id=11941&id=11966&id=11929&id=11914&id=11965&id=11927&id=11960&id=11946&id=11925&id=11952&id=11968&id=11943&id=11967&id=11930&id=11926&id=11917&id=11934&id=11920&id=11945&id=11921&id=11962&id=11933&id=11928&id=11939&id=11954&id=11957&id=11935&id=11956&id=11953&id=12025&id=11984&id=11999&id=12029&id=11983&id=12004&id=12031&id=11972&id=12003&id=12000&id=12026&id=11982&id=11975&id=12008&id=11987&id=11979&id=12019&id=11980&id=12016&id=12028&id=12018&id=11989&id=11990&id=11986&id=12013&id=11988&id=12011&id=12006&id=12022&id=11991&id=12024&id=11973&id=12065&id=12058&id=12048&id=12063&id=12067&id=12089&id=12082&id=12049&id=12052&id=12053&id=12075&id=12045&id=12064&id=12094&id=12078&id=12057&id=12051&id=12070&id=12087&id=12090&id=12035&id=12059&id=12066&id=12038&id=12041&id=12068&id=12050&id=12092&id=12037&id=12076&id=12044&id=12055&id=12056&id=11426&id=12116&id=12103&id=11431&id=11428&id=12105&id=11433&id=12098&id=12122&id=12099&id=12101&id=11436&id=11422&id=11435&id=11418&id=12106&id=12108&id=11420&id=12118&id=12095&id=11412&id=12114&id=12126&id=11430&id=12119&id=12107&id=11424&id=12096&id=11432&id=11409&id=12125&id=11421&id=11457&id=12142&id=11459&id=11444&id=11395&id=11381&id=11460&id=11406&id=11462&id=12129&id=11382&id=12137&id=11404&id=11449&id=12136&id=11385&id=11453&id=11386&id=11393&id=12144&id=11390&id=11445&id=11383&id=11443&id=11402&id=11379&id=12130&id=11380&id=12132&id=11448&id=11407&id=12135&id=12184&id=12179&id=12162&id=12161&id=12191&id=12157&id=12155&id=12146&id=12199&id=12151&id=12205&id=12196&id=12197&id=12158&id=12198&id=12186&id=12173&id=12204&id=12183&id=12182&id=12168&id=12185&id=12153&id=12207&id=12152&id=12172&id=12175&id=12194&id=12163&id=12147&id=12166&id=12165&id=12195

Change History

Changed 3 years ago by work@…

  • owner changed from work@… to robert@…
  • status changed from new to assigned

It seems like you are reaching  mod_jk / tomcat / apache's header limits...

While we could of course try to dig through configuration settings to increase the buffer size, it's probably better to not reach the buffer limit at all. Could you rewrite that POST request to a shorter version? Otherwise we have to keep increasing the buffer limit to accomodate the growth of the platform, and that's probably not ideal...

It seems there is server setting solution:

8192 is the default packetSize option for the AJP <Connector>. It
appears that you are trying to shove more data into a single packet than
is possible.

It's not your fault. Well, sort-of, but not really. Your code is
probably fine.

I believe that either the AJP protocol or Tomcat's implementation of it
requires that all of the HTTP headers fit into the first AJP packet sent
back to the web server. So, if you have more than 8192 bytes of headers
to send back to the web server, you'll get this error. That's what I
asked about the large cookie values. It also happens when you use SSL
with proxied certificate chains which can eat up a lot of space, too.

Anyhow, there is good news: you can change the default packetSize like this:

<Connector ... packetSize="16384" />

MAKE SURE YOU SET THE max_packet_size IN mod_jk TO BE THE SAME VALUE,
otherwise everything will go to hell. You can go up to 65536 bytes for
the packet size if you really need to. Note that larger packet sizes may
have a performance impact.

Another option would be to limit the total size of cookie information
you allow your custom JSP to emit. It's not foolproof (you can always
use some other header to reach this limit) but it might help if you are
not comfortable increasing the AJP packet size for some reason.

 source

Changed 3 years ago by robert@…

  • owner changed from robert@… to work@…

The headers being sent are not very large:

POST /gscf-0.8.0-dbnptest/advancedQuery/performAction HTTP/1.1
Host: test.dbnp.org
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:2.0) Gecko/20100101 Firefox/4.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Referer: http://test.dbnp.org/advancedQuery/search?criteria.0.entityfield=Study.title&criteria.0.operator=contains&criteria.0.value=antiresdev&operator=and&entity=Sample
Cookie: [300 bytes of cookies, removed for privacy reasons]

These headers are similar when selecting only 4 samples, instead of 500. It is only the POST body being quite large. That shouldn't be a problem.
Also, the post parameters can hardly be rewritten into a shorter string, keeping in mind that the number of samples might be larger in the future.

Changed 3 years ago by robert@…

The data was sent using POST to a GSCF method, but that method sent the data using GET to the modules. This is resolved now in r1798.

Changed 3 years ago by work@…

  • status changed from assigned to closed
  • resolution set to fixed
Note: See TracTickets for help on using tickets.