Our assumptions are incorrect because Bris Milah is a big deal. Talmudic sources talk of why Bris Milah is such a big deal (Nedarim 31b-32a). (Look it up for the many details)
The Midrash tells us in Pirkei D'Rabi Eliezer 28
There are other reasons, of course, for celebrating, but this will suffice for now (it was inspired by the teaching I found last week (presented two posts ago)).
ר' אומ' לא עכב אברהם מכל אשר צוהו, שנ' ובן שמנת ימים, וכשנולד יצחק בן שמנת ימים הגיש למילה שנ' וימל אברהם את יצחק בנו בן שמנת ימים, מכאן את למד שכל איש שמגיש בנו למילה, כאילו כהן גדול מקריב מנחתו ונסכו על גבי המזבח, מכאן אמרו חכמים חייב אדם לעשות שמחה ומשתה באותו היום שזכה להמול את בנו כאברהם שמל את בנו, שנ' וימל אברהם את יצחק בנו וכו'
Translation: Avraham did not neglect anything God commanded him to do, as it says "And when he is eight days old [you will circumcise him]..." When Yitzchak was born, he was brought to his bris at eight days of age, as it says "And Avraham circumcised his son when [his son] was eight days old."
We learn from here than anyone who brings his son to his bris is compared to the High Priest bringing a Mincha offering on the altar.
From this comparison the wise men taught that a person must rejoice and make a celebratory meal on the day he merits to have his son circumcised, as it says with regard to Avraham "That Avraham circumcised his son" [and the implication is that he made it into a big deal.]