While refusal is your right, employers might limit certain roles or tasks requiring health assessments for safety reasons. Discuss your concerns with the employer or seek alternative options.