Chief Lee Dixon, was responding to a report of an armed man Thursday night around five when he was shot.
Many people in Olney said they remember him from high school, or knew his family.
Olney Police Chief Barry Roberts says when a small community loses a police chief, it is very traumatic.
"He was tickled to death, doing a great job, and everybody in town was glad to have him back. I was glad to have him back. We had just had coffee this morning. Now this happened and it's just unreal," Mayor Ronnie White of Little River-Academy, said Thursday night after the shooting.
Mayor Ronnie White says Chief Dixon was a one-man force.
He had only been serving the town of Little River-Academy for one month before he was shot and killed.
Police say when asked if he needed back up on the call, he said no, and then when he needed it, it was too late.
"He went to the call and called for back up and the young man shot him with a shot gun," says Chief Roberts.
It's something Roberts says is heart breaking for a community.
"A police chief is many times like the town minister and he's a leader in the community. He is well known and has good rapport with the folks so it be a tragedy at the very least and then of course when the person at the head of the ship is gone you know you gotta go through the whole process of trying to pull the troops together," Roberts says.
Roberts says it's times like these when people should remember the risk these men take every day.
"We live on a day to day basis of life and death you know we get up and go to work and we don't know if we're going to come home or not," Roberts says.
Dixon's ex wife, Melanie Dixon, said Dixon loved his family and loved his job.
She says he died doing and protecting what he loved.
Dixon's body will be sent to Dallas for an autopsy.
Copyright 2016 Nexstar Broadcasting, Inc. All rights reserved. This material may not be published, broadcast, rewritten, or redistributed.