Educational Video Platform with a GPT Recommendation Engine
Shared Google Drive: https://drive.google.com/drive/folders/1iQphjeupXE0J5FsgBP2wPTk_dsTTGSHW
Client: Riordan Alfredo
Description: A nursing simulation app which plays simulation videos, tracks students' locations during the simulation, and opens quizzes about the video content.
- Students and teachers are authenticated to their accounts in the app.
- Students and teachers are enrolled into units
- Teachers can upload videos, create assessments and view analytics on the enrolled students
- Students can watch videos, sit assessments and get additional visualizations from videos and lectures
- Students can receive AI assistance for their assessment results as well as during videos
- Captions are automatically generated on video upload via OpenAI Whisper
This project is structured as a monorepo with a backend server under /api and a frontend server under /frontend. We additionally have a server under /whisper that hosts our OpenAI Whisper model and interface.
The backend component consists of 2 parts: a Postgres database and a TypeScript Express application that exposes API endpoints for the frontend to consume. This is continuously deployed on Railway.
The frontend component is a TypeScript Next.js web application containing the user interface and logic for fetching from the backend. This is continuously deployed and hosted on Amplify.
The whisper component is a Python Flask server that exposes some API endpoints. In the backend, an open-source version of OpenAI Whisper is running that performs our video transcription.
.
├── api
│ ├── postgres
│ │ ├── data
│ │ └── Postgres Dockerfile
│ └── src
│ ├── persistence
│ └── routes
│ ├── Auth
│ ├── Quiz
│ ├── User
│ └── Video
├── frontend
│ ├── components
│ │ ├── Carousel
│ │ ├── Login
│ │ ├── Quiz
│ │ │ ├── QuizCreation
│ │ │ └── Results
│ │ ├── Sidebar
│ │ ├── Uploads
│ │ └── Visualization
│ ├── pages
│ │ ├── api
│ │ └── unit
│ ├── public
│ ├── service
│ │ └── S3 & Video Services
│ ├── styles
│ └── utils
├── tools
│ ├── docker
│ │ └── Docker-Compose Files
│ └── Start & Stop Scripts
├── whisper
│ └── Python Whisper Transcription Server
Prerequisite: Docker Desktop
The easiest way to begin development is to run the shell scripts under /tools which will setup Docker containers that run components of the project. Dockerfiles and docker-compose files have been configured for each of our services so that environment variables are configurable and the services integrate with each other locally by default.
| Script | Services |
|---|---|
/start.sh |
Starts all 3 primary components: Database, API and Frontend |
/start_backend.sh |
Starts backend components: Database and API |
/start_database.sh |
Starts only the database |
/stop.sh |
Stops any component started by the scripts above |
/restart.sh |
Runs stop.sh and then start.sh for quick restarting |
/start_whisper.sh |
Starts the Whisper backend server (separated due to size) |
/stop_whiser.sh |
Stops the Whisper backend server |
OAuth is an authorization framework that enables apps to obtain limited access to user data. Using this service, users can login to the Edvantage platform if they are a verified Monash student.
Using OAuth 2.0 requires setting up credentials through Google API Console.
- Navigate to Credentials and create a new project.
- Choose a project name and the organisation (choosing monash.edu will enable access to Monash users only).
- Under “Create Credentials” choose “OAuth Client ID”.
- Choose “Internal” for the User Type to limit who can login.
- Fill in basic project information.
- For the “Authorised JavaScript Origins” and “Authorised Redirect URIs” sections include all the URIs for production, testing and development including port numbers.
- Note the Client ID and Client Secret.
After these steps are followed, the authentication service is ready to be used in the client by updating the value of clientID in /frontend/pages/_app.tsx
We call various API's through our application, such as AWS S3 and OpenAI ChatGPT and, to do this, we need to have generated API Keys. We also need API keys implemented in our applications to securely access our backend servers.
| API Key | Used In | Local Development | Production | Notes |
|---|---|---|---|---|
| AWS Key & Secret | Frontend, Whisper | ./frontend/.env.local: NEXT_PUBLIC_AWS_ACCESS_KEY, NEXT_PUBLIC_AWS_SECRET_KEY ./tools/docker/docker-compose-whisper: AWS_ACCESS_KEY, AWS_SECRET_KEY |
./frontend/.env.production: NEXT_PUBLIC_AWS_ACCESS_KEY, NEXT_PUBLIC_AWS_SECRET_KEY ./tools/docker/docker-compose-whisper: AWS_ACCESS_KEY, AWS_SECRET_KEY |
An environment variable approach is suitable for whisper, however long-term the API keys should be moved out of the frontend and into the backend application. |
| OpenAI API Key | Frontend | ./frontend/.env.local: NEXT_PUBLIC_OPENAI_API_KEY | ./frontend/.env.production: NEXT_PUBLIC_OPENAI_API_KEY | The API calls currently occurring in the frontend should be wrapped in backend calls that are secured by our API key, and the OpenAI key will be stored in the backend, via env vars or AWS KMS. |
This project requires the following software:
- Node.js (v18 or above)
- Docker (For building and running the project containers)
- PostgreSQL (Connectable through DBeaver or any other SQL client)
- Python (v3.8 or above, required for the Whisper component)
- Accounts required for deployment (Railway, AWS)
- Production URL's
- CI/CD Builds and any notes
- Release processes
To deploy this project, you will need to:
- Set up the environment variables as described in the API Keys section for local or production environments.
- For local testing and development, use the provided Docker scripts under
/tools. - For production, the project is continuously deployed via AWS CodePipeline and AWS Elastic Beanstalk for the backend and AWS Amplify for the frontend. Ensure the proper setup of these platforms and configure the CI/CD pipelines accordingly.
The default database credentials are as follows:
- Host: 127.0.0.1
- Port: 5432
- Database: edvantage
- User: admin
- Password: Password
These are the descriptions and versions of our primary dependencies.
| Package | Description | Version |
|---|---|---|
| aws-sdk | Provides client for AWS API's and services | 2.1423.0 |
| cors | Provides CORS functionality for API calls | 2.8.5 |
| express | A backend JS framework for building simple API's | 4.18.2 |
| google-auth-library | Used for creating OAuth functionality | 8.8.0 |
| pg | Package that provides a pre-built Postgres client | 8.5.1 |
| Package | Description | Version |
|---|---|---|
| @chakra-ui/react | React CSS styling package | 2.5.5 |
| @react-oauth/google | React integration with Google OAuth | 0.11.0 |
| aws-sdk | Provides client for AWS API's and services | 2.1423.0 |
| chart.js | Statistics library used to create visualisations | 4.4.0 |
| next | Server-side front-end framework that extends React | 13.3.0 |
| openai | Package for integrating with OpenAI API's | 4.1.0 |
| pg | Package that provides a pre-built Postgres client | 8.5.1 |
| react | All-in-one front-end framework | 18.2.0 |
| react-dom | Extension to react that makes DOM easier to work with | 18.2.0 |
| typescript | Typed extension to the Javascript programming language | 5.0.3 |
| Package | Description | Version |
|---|---|---|
| flask | A lightweight backend framework for building API's | latest |
| openai-whisper | Open-source whisper transcription model | latest |
| boto3 | Provides client for AWS API's and services | latest |
| flask_cors | Extension to flask that fixes CORS | latest |
A list of common issues encountered during development and ways to get around them, as well as useful documentation.
- Logging locations for troubleshooting
- Frontend logs - Will generally appear in the browser console, will appear in the frontend docker container on a major error that crashes the application
- Backend logs - Will appear in the backend docker container, check here first if receiving an error when calling an internal API endpoint
- Postgres logs - Will appear in the postgres docker container, check here if performing an action that interacts with the database and fails unexpectedly
- Whisper logs - Will appear in the whisper docker container, check here when transcription is failing
- "frozen-lockfile" issue - When building the frontend container, an error will come up like "failed to solve process" or something about the yarn lockfile
- Solution - Remove the "--frozen-lockfile" section from line 12 of frontend/Dockerfile, and move the semicolon to the end of "yarn"
- Docker Desktop - Required for local development
- Postman - Useful for sending REST requests to your backend servers without the need for frontend integration
- DBeaver - Useful for connecting to our Postgres database and interacting with/modifying the data
We use Semantic Versioning for this project. For future releases, please adhere to this structure:
- MAJOR version when you make incompatible API changes,
- MINOR version when you add functionality in a backward-compatible manner, and
- PATCH version when you make backward-compatible bug fixes.
Remember to update the version in the package.json file as well as tag the commit with the new version.
For developers intending to contribute to the project, please follow these steps:
- Fork the repository to your GitHub account.
- Create a new branch from the 'main' branch with a descriptive name about the feature or fix.
- Implement your changes and write clear, understandable commit messages.
- Push your branch to your fork.
- Open a pull request against the 'main' branch of the original repository.
- Describe the changes in the pull request description and mention any issue(s) your PR addresses.
Please ensure your code adheres to the existing style guidelines and passes any tests. The project maintainers will review your PR and provide feedback.