With the proper adhesive, I would totally glue a mount to the brick. But only if it was going to be absolutely permanent. Best to use a mount with post leveling adjustment......just in case. They have adhesives that will hold a car to a billboard. Sticking a tv to a fireplace would be easy peasy. Such adhesives are likely not cost efficient and would require some hefty safety precautions. Not worth the effort or cost when appropriate fasteners are readily available.
"When I eat, it is the food that is scared." - Ron Swanson