RE: Japitools 1.5 support tough design issue

[David Holmes]

Stuart,

> It appears that what actually happens is that the foo(String) method
> of Sub1 actually gets compiled as if it were:
> String foo(Object o) {foo((String) o);}
> String foo(String str) {...}
> I assume that the foo(Object) method is one of those "bridge" methods
> that we're currently ignoring in japitools.

Yes a bridge method will typically be generated to meet the requirements of
JLS 15.12.4.5. Such methods will be marked as synthetic.

You need to remember that generic types, eg. T, will be replaced by their
erasure in the bytecode and that the compiler will then introduce bridge
methods as needed to cause casts to the right "T" to occur.

> class Super<T>
> {
>   String foo(T t) {return "Super";}
> }

So class Super has the method:

    String foo(Object t);

> class Sub1 extends Super<String> {
>   String foo(String str) {return "Sub1(String)";}
> }

Sub1 logically has a single method:

    String foo(String t)

which is inherited from Super<String> and overridden in Sub1. But the
bytecode physically has two methods:

    String foo(Object);
    String foo(String);

and the compiler "overrides" the Object version with a bridge method that it
casts to String and invokes the String version - thus ensuring only the one
implementation actually gets called from the callers perspective.

> // Sub2 is a compile error: "name clash: foo(java.lang.Object) in Sub2
> and foo(T) in Super<java.lang.String> have the same erasure, yet
> neither overrides the other"
> //class Sub2 extends Super<String> {
> //  String foo(Object str) {return "Sub2(Object)";}
> //}

Right. The method you try to add as an overload of foo in Sub2 has the same
signature as the erasure of the Super.foo method - hence a compile-time
error.

> // Sub3 is a compile error: "name clash: foo(java.lang.Object) in Sub3
> and foo(T) in Super<java.lang.String> have the same erasure, yet
> neither overrides the other"
> //class Sub3 extends Super<String> {
> //  String foo(String str) {return "Sub3(String)";}
> //  String foo(Object str) {return "Sub3(Object)";}
> //}

Same problem.

> // Sub4 is a compile error: "foo(java.lang.String) in Sub4 cannot
> override foo(T) in Super; attempting to use incompatible return type"
> //class Sub4 extends Super<String> {
> //  void foo(String str) {System.out.println("Sub4(String)");}
> //}

Sub4.foo differs from the logical form of Super.foo only in return type.
That is only permitted if the return type is covariant with the original
which it isn't. So again an error.

> class Sub5 extends Super<String> {
>   void foo(Object str) {System.out.println("Sub5(Object)");}
> }

Logically you have:
     String foo(String) from Super
     void foo(Object)   from sub5

That is okay as they have different signatures. Physically you have three
methods in the bytecode:
     String foo(Object);
     String foo(String);
     void foo(Object);

This is okay the VM knows how to handle this. The VM's rules are less strict
than the languages. But it is hard to grok. The compiler will choose the
most specific form of the method to invoke based on the type of the argument
actually passed - so foo("aString") will always invoke the "String
foo(String)" form.

> // Sub6 is a compile error: "foo(java.lang.String) in Sub6 cannot
> override foo(T) in Super; attempting to use incompatible return type"
> //class Sub6 extends Super<String> {
> //  void foo(String str) {System.out.println("Sub6(String)");}
> //  void foo(Object str) {System.out.println("Sub6(Object)");}
> //}

Same error as Sub4.

> // Sub7 is a compile error: "foo(java.lang.String) in Sub7 cannot
> override foo(T) in Super; attempting to use incompatible return type"
> //class Sub7 extends Super<String> {
> //  void foo(String str) {System.out.println("Sub7(String)");}
> //  String foo(Object str) {return "Sub7(Object)";}
> //}

Same again.

> class Sub8 extends Super<String> {
>   String foo(String str) {return "Sub8(String)";}
>   void foo(Object str) {System.out.println("Sub8(Object)");}
> }

Okay again. An override of the logically inherited method, plus a new
method.

>     // Prints Sub1(String)
>     try {System.out.println(s1.foo(""));} catch (Exception e)
> {System.out.println(e);}

Yep that's okay.

>     // ClassCastException
>     try {System.out.println(s1.foo(new Object()));} catch (Exception
> e) {System.out.println(e);}

Yep Sub1's bridge method takes care of this.

>     Super<String> ss1 = new Sub1();
>     // Prints Sub1(String)
>     try {System.out.println(ss1.foo(""));} catch (Exception e)
> {System.out.println(e);}

Yep okay.

>     Super s5 = new Sub5();
>     // Prints Super
>     try {System.out.println(s5.foo(""));} catch (Exception e)
> {System.out.println(e);}

Ack! Unchecked warning - you have a raw Super object.

>     // Prints Super. Really interesting that this *isn't* a
> ClassCastException.
>     try {System.out.println(s5.foo(new Object()));} catch (Exception
> e) {System.out.println(e);}

A raw Super object has a String foo(Object) method and that is what you have
invoked. So the signature used was for the Object version that exists in
Super and is not overridden. See below.

> What do I conclude from these results?

Pay attention to unchecked warnings. :-)

> Furthermore Sub5 is interesting because it ends up with two methods
> foo(Object), which differ only in return type. The one with return
> type String is only accessible when used as an unqualified Super. In
> other words the compiler probably doesn't actually insert a String
> foo(Object) method here at all, it's just found at runtime by
> superclass lookup.

The call resolves to invoking "foo(Object)" which exists in Super. The Sub5
class doesn't have a bridge method as it doesn't override a "compatible"
foo. If the Super reference was a Super<String> reference then the compiler
would refuse to compile the call. This sort of weirdness is what you get
with raw types. The compiler doesn't generate a bridge method because it
doesn't need to "redirect" foo(String) calls to the subclass implementation,
and it "knows" that it won't allow foo to be called with anything but a
String. But using the raw type subverted the compile-time check and so the
super method was called with a non-String argument.

> Sub1 has the following methods:
> 1A) String foo(Object) (BRIDGE)
> 1B) String foo(Object but instantiated as String) (inherited from Super)
> 1C) String foo(String) (declared directly)

There are two foo methods in Sub1:
   foo(Object) overrides Super.foo(Object) to perform the bridge function
   foo(String) "overrides" the logical Super.foo(String) method that was
actualy implemented by foo(Object).

> Sub5 has the following methods:
> 5A) String foo(Object but instantiated as String) (inherited from Super)
> 5B) void foo(Object) (declared directly)
> 5C) (maybe?) String foo(String) (BRIDGE?) (automatically added by
> compiler? I can't think of a straightforward test for whether this
> actually exists or not)

Sub5 has:
  void foo(Object) declared
  String foo(Object) inherited
but the compiler will only let "String foo(Object)" be invoked with a String
argument.

The different return types here really make things confusing I agree, but
the rules for determining which method to invoke will always choose the most
specific method.


> Sub8 has the following methods:

I think you get the picture. Look at the output from javap -c to see what
the compiler produces. Or use some other clasfile editor if you can't touch
Sun tools.

The problem is that different compilers may be able to enforce the rules in
different ways and so unchecked usages may result in different behaviour
depending on the compiler - I'm not 100% sure.

> The goal is to try to compare these in a way that's meaningful both
> from a 1.4 perspective and a 1.5 perspective. However...

I don't quite follow the problem here, in that if you are processing a
parameterized 1.5 type then you need to apply the 1.5 rules regarding
overloads, overrides, covariant returns and allow for bridge methods (which
should be marked as such).

You need to know whether you are applying 1.4 or 1.5 rules regardless, to
deal with covariant returns and varargs.

Hope this helps. Generics are so much fun - NOT! :-)

Oh a final warning: javac at least still contains many bugs with regard to
generics. What you observe need not be what the language truly allows or
prohibits. Can't speak for the other compilers.

Cheers,
David Holmes

Shameless plug: co-author of "The Java Programming Language 4th Edition"
(and  3rd) with Ken Arnold and James Gosling.