So thanks for the link Lars and it explains why we didn't see a performance improvement with a secondary index.
BW had generated a concat attribute for the second table, with the primary key of the first table. Very clever, and another example of how HANA separates intent from optimization.