To get hired, you have to pass all steps of the process.  This could include any or all of the following depending on the job:  the initial application and resume review, interview, aptitude testing, medical exams, drug testing, physical fitness test for positions like being a federal agent, security clearance background check, psychological exams, the suitability review, and yes....the polygraph.

An unfavorable result in any of the hiring steps will cause the applicant to get that thin rejection letter in the mail.  No matter how "innocent" the applicant claims to be.

The polygrapher doesn't make the final call during the polygraph phase.  The applicant's polygraph charts and session are reviewed by someone else who says yay or nay.  But these people have about as much power as the other hiring officials who review the applicant's results during the other steps.