Description of the need
Theme Debug is a popular option for any themer. With the devel module, one can turn it on in core, but without the devel module a user must edit the 'system.core.json' config file.
Add the Theme Debug config setting to: admin/config/development/?? (maybe: admin/config/development/logging)
Alternatives that have been considered
It's not clear to me that is necessary in core. On occasion it would be really useful. Usually, when I'm working on core/contrib issues with a basic install and devel module is not available. My own sites all have the devel module enabled and this is not an issue.
Is there a contributed module that accomplishes this?
Yes, the devel modules solves this and most anyone theming their site is probably using devel.
But, I can imagine use cases when a user is not creating a full theme and does not have devel installed, would like to see which templates are being used. I am currently thinking about it, because of the sub-theming instructions that tell users to edit the "system.core.json" file, since we can not assume that Devel is available. It would be so much easier if Theme Debug were in the UI in core. :-)
I'm assuming that this idea has been discussed before and probably dismissed for good reason. But, I can't find a ticket for it. I am sure someone else will have this idea in the future and let's either have an active ticket for them or a closed ticket with a clear and concise explanation about why this is a bad idea.