Know Your Own Fish Tank By Looking at Someone Else