Dirk,

I don't know to what extent failing a DoD polygraph might adversely affect future DoD contract opportunities, but declining the polygraph would preclude you from receiving clearance for the Special Access Program that your new job requires.

Note, however, that the DoD counterintelligence-scope polygraph program failure rate is very low. It seems that although many may be initially told they had "problems" with a question and be asked for an explanation, ultimately the only ones who "fail" are those who make "substantive" admissions. See the subchapter on the DoD Polygraph Program at pp. 57-59 of the 3rd edition of The Lie Behind the Lie Detector and also the discussion thread, How to Pass the DoD CI-Scope Polygraph.