problem executing demanding DELETE query

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

problem executing demanding DELETE query

René Verheij
Hello Sesame community,

I have a DELETE query which if executed correctly would remove about 3 million triples of the 6 million triples in total.

DELETE { ?s ?p ?o } WHERE { ?s rdf:type abg:Aangeboden-Product. ?s ?p ?o. }

Over HTTP I got a connection timeout errors, so I went to the Sesame Console over SSH and retried the query which now gave me this error:

tomcat ava.lang.OutOfMemoryError: Java heap space

So I googled a bit and changed the memory available to tomcat6 by creating /usr/share/tomcat6/bin/setenv.sh

export JAVA_OPTS="-Xms256m -Xmx4096"

.. the query runs much longer now, but still throws this error:

 java.lang.OutOfMemoryError: GC overhead limit exceeded

I tried adding LIMIT 1000 after the WHERE { } part, but that's not a valid query. 

I will try to get the same result for now by using a several DELETE queries that match smaller amounts at the time, but for the future, I would like to know if there is anything that I could do to still execute these kind of queries directly.

Tnx in advance,

René

------------------------------------------------------------------------------
Android is increasing in popularity, but the open development platform that
developers love is also attractive to malware creators. Download this white
paper to learn more about secure code signing practices that can help keep
Android apps secure.
http://pubads.g.doubleclick.net/gampad/clk?id=65839951&iu=/4140/ostg.clktrk
_______________________________________________
Sesame-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sesame-general
Reply | Threaded
Open this post in threaded view
|

Re: problem executing demanding DELETE query

Jeen Broekstra

This kind of update should normally run smoothly for the data sizes
you're talking about.

I am taking a look right now. Which version of Sesame are you testing
with, what store, and what store configuration (indexes, inferencers, etc.)?

Jeen

On 2/11/13 11:28 pm, René Verheij wrote:

> Hello Sesame community,
>
> I have a DELETE query which if executed correctly would remove about 3
> million triples of the 6 million triples in total.
>
> DELETE { ?s ?p ?o } WHERE { ?s rdf:type abg:Aangeboden-Product. ?s ?p ?o. }
>
> Over HTTP I got a connection timeout errors, so I went to the Sesame
> Console over SSH and retried the query which now gave me this error:
>
> tomcat ava.lang.OutOfMemoryError: Java heap space
>
> So I googled a bit and changed the memory available to tomcat6 by
> creating /usr/share/tomcat6/bin/setenv.sh
>
> export JAVA_OPTS="-Xms256m -Xmx4096"
>
> .. the query runs much longer now, but still throws this error:
>
>   java.lang.OutOfMemoryError: GC overhead limit exceeded
>
> I tried adding LIMIT 1000 after the WHERE { } part, but that's not a
> valid query.
>
> I will try to get the same result for now by using a several DELETE
> queries that match smaller amounts at the time, but for the future, I
> would like to know if there is anything that I could do to still execute
> these kind of queries directly.
>
> Tnx in advance,
>
> René
>
>
> ------------------------------------------------------------------------------
> Android is increasing in popularity, but the open development platform that
> developers love is also attractive to malware creators. Download this white
> paper to learn more about secure code signing practices that can help keep
> Android apps secure.
> http://pubads.g.doubleclick.net/gampad/clk?id=65839951&iu=/4140/ostg.clktrk
>
>
>
> _______________________________________________
> Sesame-general mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/sesame-general
>


------------------------------------------------------------------------------
Android is increasing in popularity, but the open development platform that
developers love is also attractive to malware creators. Download this white
paper to learn more about secure code signing practices that can help keep
Android apps secure.
http://pubads.g.doubleclick.net/gampad/clk?id=65839951&iu=/4140/ostg.clktrk
_______________________________________________
Sesame-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sesame-general
Reply | Threaded
Open this post in threaded view
|

Re: problem executing demanding DELETE query

Jeen Broekstra
On 4/11/13 11:03 am, Jeen Broekstra wrote:
>
> This kind of update should normally run smoothly for the data sizes
> you're talking about.

Slight update. Sesame currently caches the result of the WHERE clause in
memory for SPARQL updates. This is necessary to be able to guarantee
proper behavior, but unfortunately causes you to experience
OutOfMemoryErrors.

We are investigating a more scalable approach but in the meantime the
only workaround is to use direct API methods for removal or work with
smaller batches.

Jeen

------------------------------------------------------------------------------
Android is increasing in popularity, but the open development platform that
developers love is also attractive to malware creators. Download this white
paper to learn more about secure code signing practices that can help keep
Android apps secure.
http://pubads.g.doubleclick.net/gampad/clk?id=65839951&iu=/4140/ostg.clktrk
_______________________________________________
Sesame-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/sesame-general