Nearly 100 people will lose their jobs when a Fairfield County factory shuts its doors later this year.