Nobody in my circle knew I owned a $550K house—and there was a reason I kept it that way.