This research study investigates the impact of using a large language model (LLM) like ChatGPT as a diagnostic aid for physicians. The study, conducted with 50 doctors, randomly assigned them to two groups: one with access to the LLM and the other with only conventional resources. The results indicate that having access to the LLM did not significantly improve the diagnostic reasoning performance of physicians compared to the control group, although the LLM alone performed better than both groups of doctors. This suggests that while LLMs have potential as tools for assisting with diagnosis, their effectiveness in clinical practice needs to be further explored and integrated more effectively.