Detecting contraband. Guard dogs will smell prisoners and items (books, shop items, sheet metal, kitchen ingredient, etc.) inside the prison. If they smell contraband they will bark and the dog handler will search the prisoner or items. Dogs can only detect smelly contraband, poison, cigarettes, booze, and narcotics.
Detecting escape tunnels. If a dog walks over an escape tunnel they will dig into the ground and a yellow flag will pop up. Be aware that false alarms are not uncommon. You will know if there is really an escape tunnel if multiple yellow flags pop up within a very short span of time. If a prisoner happens to be digging the tunnel when the dog detects it, the dog will pull him up to the surface. It also seems that false alarms will always be in the same places, just very far apart time-wise.
Chasing escaping prisoners. If a prisoner is escaping, the dog will be let off of its leash and it will chase after the prisoner. The dog can run much faster than prisoners and guards.
General prisoner management. If prisoners are rioting or fighting, guard dogs will attack them. However, as of Alpha 35, guard dogs will sometimes bite even an unconscious prisoner to death if a guard doesn't get to the prisoner in time.
If a dog's dog handler is killed, the dog will bite the prisoner that killed their owner.