Closed Bug 526143 Opened 15 years ago Closed 12 years ago

Static analysis for "use-after-invalid" bugs

Categories

(Developer Infrastructure :: Source Code Analysis, enhancement)

x86_64
Linux
enhancement
Not set
normal

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: cjones, Unassigned)

References

Details

My use case is that I have a class like the following

  class Foo {
  public:
     bool m1();
     bool m2();
     // ...
  }

The |bool| return code means true=success/false=error.  But in my case, |false| implies a "Foo instance fatal" error; i.e., calling any other of the public methods of Foo becomes illegal (or if you prefer, useless or redundant, since one error implies all future calls to Foo methods will return the same error).  IOW, if this happens, there's a bug in the client's error handling code.  For example

  Foo *f = new Foo();
  bool ok = f->m1();  // ok = false
  // ok == false ==> f is "dead"
  f->m2();  // ERROR!  f is dead

It's easy to check this property dynamically, with an isValid member variable or something, and an |assert isValid| check at the top of each method.  (That's ugly, but in my case that code would be automatically generated so it's not a big deal.)  In fact, for my use case, I get this dynamic check for "free".

Another dynamic analysis is to have the Foo methods randomly fail, along with the isValid check above.  I may use this approach too.

But I think this makes for a nice and not too complicated a static analysis, and it's a slam dunk for ESP if that's needed.  We would annotate Foo::m1/m2 with |attribute((instance_fatal))| or something, and the analysis would prove that at instance_fatal method calls, the instance wasn't yet "dead".  To avoid dealing with pointer aliasing, the analysis could assume that each distinct pointer variable (including |this|) represented a distinct instance (or NULL).  That assumption makes the analysis unsound, but shouldn't be a big deal in practice.

For starters, this could be context-insensitive and intra-procedural.  If necessary, it could pretty easily make use of ESP to get context sensitivity.  It'd be fun to use the callgraph, but for that amount of investment doing an analysis like that of bug 512868 would probably be more useful for my use case.
(In reply to comment #0)
> To avoid
> dealing with pointer aliasing, the analysis could assume that each distinct
> pointer variable (including |this|) represented a distinct instance (or NULL). 
> That assumption makes the analysis unsound, but shouldn't be a big deal in
> practice.
> 

It'd be even easier to make the simplifying assumption of one instance per Type, i.e. all Foo* variables in a method point at the same Foo.  Then modulo crazy inheritance and pointer hackery, the analysis would be sound.  If that gave too many false positives we could move to the instance-per-variable assumption.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → WONTFIX
Product: Core → Firefox Build System
Product: Firefox Build System → Developer Infrastructure
You need to log in before you can comment on or make changes to this bug.