no "recycling" for things that are still referenced

The following problem is describe by David http://www.cs.nyu.edu/pipermail/smt-comp/2014/000294.html

> One of the solvers was 'recycled' in StarExec,
> causing all job-pairs with that solver to fail from that time on.

I was assuming that as long as there's a reference to something, then it cannot be garbage-collected.

Assuming the semantics of "user X recycles ressourc Y" is "remove all reference to Y that were created by X"; while other references (e.g., made by the space leader) remain in place (and thus keep Y alive).

Such information is vital for safely running (and planning) competitions. I was asking about this earlier
https://starexec.forumotion.com/t58-immutability-of-data-referenced-by-ids-semantics-of-deletion-recycling

@David: your emails about SMTCOMP execution are very instructive (for me, planning to run Termination).

Johannes,

There is a problem with not recycling primitives that still have an outstanding reference. Suppose I have some public benchmarks or solvers, and you link them in your space. If we were to implement the approach you are suggesting, now I can no longer delete them unless you unlink them. You can mount a denial-of-quota attack on my account this way. Of course, we don't expect malicious behavior, but it could happen unintentionally. That is why we designed the system so that a user can recycle primitives (benchmarks, solvers, or jobs) even if others have references to them. If you want to have a copy of a benchmark or solver and be sure it cannot disappear out from under you, then you have to copy it, not link it.

Aaron

Hi Aaron,

this depends on what counts towards your quota. It could be defined in such a way that I (the "attacker") will be billed in case I have the last reference to a primitive.

How do other OS handle this? Some research turned up this (Section 21.1.5)

> Whenever you delete a primary link, the most recently created hard link automatically
> becomes the new primary link. At that time, the file’s size is subtracted from the space usage
> reported on old primary parent directory and added to the space usage reported
> on the newly assigned primary parent directory.

http://www.novell.com/documentation/oes/nss_enu/?page=/documentation/oes/nss_enu/data/btxqabr.html

I like this solution. The potential "quota battles" seem far less problematic to me than the current situation in which benchmarks and solvers can disappear with no warning. As I see it, the quotas are around mostly just to make sure no one inadvertantly fills the disk or something. I don't really expect legitimate uses of the system to lead to significant quota issues. And if they do, perhaps we can all take up a collection to buy more disk space - disks are cheap!

If this solution does get implemented at some point, I would like to ask that the policy be followed *even if* it results in a user using more than their quota. Otherwise, you still have the problem of solvers/benchmarks disappearing but much more unpredictably which I think would be even worse.

A reasonable thing to do in this case (according to me) is to have the system notify the admin who should redistribute quotas in a reasonable way. Hopefully, such cases are rare.

I see the merits of the scheme Johannes mentioned. But I am a little nervous about trying to get this right in the implementation. What we have now is a cruder but much simpler to implement system:

-- link primitives if you have reason to think they are not going anywhere (e.g., SMT-LIB) and don't want to burn your quota on them.

-- copy primitives otherwise.

I agree that running out of disk is not the major concern here -- though note that contrary to what Clark was saying, our storage solution is not cheap. In fact, it cost more than my first condo (over $100k)! :-) There is a pretty big difference between a hard drive for your laptop/desktop and a NetApp, in terms of reliability and functionality.

Currently I handle requests for more quota by granting all of them, usually giving much more than was requested. So far this has not taken too much of my time. If it gets bad, we might implement a scheme where community leaders are sent these requests instead.

Aaron