Love reading this and learning about your experiments with Custom GPTs and Assistants! I understand that they have different use cases and deployment methods, but I’m curious... have you noticed any significant differences in behavior or functionality between the two? Also, have you worked document retrieval at all?
I’m working with students on experimenting with chatbots that have access to large corpora, such as hours of recorded transcripts from class lectures and video tutorials. One challenge I’ve encountered is understanding the system’s limits and actual behavior. How much source material is too much or too little? When is the chatbot actually searching through the documents versus relying on the model’s "pre-existing" knowledge? What is going on underneath the hood is a bit opaque to me, and I’ve struggled to achieve consistent results.
Hi Daniel! Thanks for your message and glad it was interesting. I find Custom GPTs and Assistants functionally equivalent if you only use file search and code generation. The biggest limitation/difference is that OpenAI hasn't added web browsing or canvas to their Assistants API, so we're kind of stymied in that regard.
My current understanding of ChatGPT's file search capability is that it really only searches the documents if explicitly instructed to or it needs to find some information that's referenced in the question. Which is to say, whatever triggers file search is currently pretty conservative. Also, fewer documents are definitely better than lots, and you shouldn't think of the search as comprehensive; rather, it'll look in documents until it finds some relevant pieces of info and then return them back, rather than search all documents and use every instance it can find in its answer.
The alternative to all this is doing your own custom embeddings, but then we're getting into pretty substantial coding and losing the benefit of relying on OpenAI to do it for us.
This is amazing! I also created a custom GPT for my AI for Film courses and I or the students interact with it throughout the class. It’s been inspiring and engaging for the students. Would love to chat more about this. Let’s find a time to zoom so I can show you the one I created and we can exchange notes.
I created a GPT I called AI Film Innovator Guru and trained it with my style (innovative, cool, professional, friendly), the tools I teach (how and why), what the workshop is about and for what audience (so it has context) and other details about AI, Film, ethics and my background. I upload pdfs of what I’ll be teaching and links to anything I’ve written about past classes. I interact with it from the very beginning of the class, teaching through practice its importance from the get-go. Because that’s how I start any big project, by creating a GPT that’s my collaborator. Throughout the classes, we interact with it like I taught it that we would. It’s been a very engaging dynamic. For my last workshop, I asked it to also have a dry sense of humor, just to make things more fun!
Love reading this and learning about your experiments with Custom GPTs and Assistants! I understand that they have different use cases and deployment methods, but I’m curious... have you noticed any significant differences in behavior or functionality between the two? Also, have you worked document retrieval at all?
I’m working with students on experimenting with chatbots that have access to large corpora, such as hours of recorded transcripts from class lectures and video tutorials. One challenge I’ve encountered is understanding the system’s limits and actual behavior. How much source material is too much or too little? When is the chatbot actually searching through the documents versus relying on the model’s "pre-existing" knowledge? What is going on underneath the hood is a bit opaque to me, and I’ve struggled to achieve consistent results.
Would love to hear if any thoughts!
Hi Daniel! Thanks for your message and glad it was interesting. I find Custom GPTs and Assistants functionally equivalent if you only use file search and code generation. The biggest limitation/difference is that OpenAI hasn't added web browsing or canvas to their Assistants API, so we're kind of stymied in that regard.
My current understanding of ChatGPT's file search capability is that it really only searches the documents if explicitly instructed to or it needs to find some information that's referenced in the question. Which is to say, whatever triggers file search is currently pretty conservative. Also, fewer documents are definitely better than lots, and you shouldn't think of the search as comprehensive; rather, it'll look in documents until it finds some relevant pieces of info and then return them back, rather than search all documents and use every instance it can find in its answer.
The alternative to all this is doing your own custom embeddings, but then we're getting into pretty substantial coding and losing the benefit of relying on OpenAI to do it for us.
Hope this is helpful!
This is amazing! I also created a custom GPT for my AI for Film courses and I or the students interact with it throughout the class. It’s been inspiring and engaging for the students. Would love to chat more about this. Let’s find a time to zoom so I can show you the one I created and we can exchange notes.
Ohhh I'd LOVE to learn more! Anything else you want to share here about what kind of custom GPT you built for your students?
I created a GPT I called AI Film Innovator Guru and trained it with my style (innovative, cool, professional, friendly), the tools I teach (how and why), what the workshop is about and for what audience (so it has context) and other details about AI, Film, ethics and my background. I upload pdfs of what I’ll be teaching and links to anything I’ve written about past classes. I interact with it from the very beginning of the class, teaching through practice its importance from the get-go. Because that’s how I start any big project, by creating a GPT that’s my collaborator. Throughout the classes, we interact with it like I taught it that we would. It’s been a very engaging dynamic. For my last workshop, I asked it to also have a dry sense of humor, just to make things more fun!
Teddy, thank you for sharing so thoroughly with us. I’m so impressed by how much you’ve developed and taught yourself, and grateful we got connected!!
Thank you very much Bailey! I'm super happy to answer any questions or brainstorm with others implementing similar solutions.