v0.3.0

What's Changed

We have added DeepSeek model support and generic support for any reasoning model from together.ai, fireworks.ai and ollama (100% local). Exposed some extra env variables for it, eventually we will improve this experience by creating Models schema in the db. We have also added support for Gmail Attachments. Also now answers will look more enhanced due to better support for markdown.

To upgrade from v0.2.4:

Due to changes in the attachment Vespa schema, we recommend you run first export the model env EMBEDDING_MODEL and then run ./deploy-docker.sh.

New Features

• Ingest Gmail PDF attachments by @junaid-shirur in #229
• Add a permission aware eval script to test our Chat function by @zereraz in #237
• Smoother deployment experience by @oindrila-b in #242
• Add Ollama support by @junaid-shirur in #267
• Add ingestion stats for OAuth by @junaid-shirur in #275
• Improve OpenAI error mapping for better messages especially for rate limits by @junaid-shirur in #279
• Add together, fireworks client and get DeepSeek (670B) to work by @zereraz in #313

Enhancements

• Enhanced font rendering with global anti-aliasing for improved visual clarity @zereraz and @shkhrg in #243
• Change History Modal to a sidebar by @zereraz in #270
• Change style of citations with layout shift as sources open by @zereraz in #271
• Make the citation title styles match and make chat title font‐medium by @zereraz in #283
• A more robust Sync engine @kalpadhwaryu in #312
• Increase default recall per document RAG by @zereraz in #261
• Improved reliability of running Vespa locally be increasing the timeouts
• More tool tips in the UI
• AWS region env now optional @junaid-shirur in #273

Bug Fixes

• Add Loader & display error inside HistoryModal UI by @kalpadhwaryu in #239
• Fix sidebar icon shift, link logo and change tooltip by @zereraz in #274
• Added missing context map for attachments by @junaid-shirur in #280
• Service account gmail worker crashed due to import of server by @zereraz in #282
• Prevent infinite loop of worker creation via cyclic imports by @zereraz in #285
• Fix parse json of LLM response to allow multiline response by @junaid-shirur in #289
• Add missing permission check for docs by @junaid-shirur in #301
• Increased memory limits in docker compose for Vespa by @zereraz in #302
• Add missing gmail attachment stats for service‐account by @junaid-shirur in #304
• Fix people and event ranking in search by @junaid-shirur in #307
• Prevent docs cleaning from affecting numeric values by @zereraz in #308
• Update attachment stats via worker message by @junaid-shirur in #311
• Fixed vite config to use .env.default in production and docker (change server’s .env.temp) by @oindrila-b in #314
• Show OAuth error toast only when an error occurs by @kalpadhwaryu in #317
• Include threadId in attachments and skip empty/password‐protected PDFs by @junaid-shirur in #318

Documentation

• More detailed docs for AI API keys section and troubleshooting related docs like re-ingestion by @oindrila-b and @zereraz

Chores

• upgrade packages via @renovate
• remove old env variables

Full Changelog: v0.2.4...v0.3.0

v0.2.4

What's Changed

• Bug Fix: LLM's would generate invalid citations like [1,2,3] -> [1] [2] [3] by @zereraz in #205
• Enhancement: Do not depend on docId in 404 errors by @junaid-shirur in #203
• Bug Fix: On slow internet chat url reload would fail by @kalpadhwaryu in #207
• Enhancement: Suppress warning logs in tests by @junaid-shirur in #206
• Enhancement: Add Readme License and Release badge @zereraz in #209 and #208
• Enhancement: A pull request template by @zereraz in #210 and #224
• Enhancement: Better logs as AI answers user query by @zereraz in #213
• Bug Fix: UI jitters during search @kalpadhwaryu in #217
• Bug Fix: Going back or forward in time for meetings and mail fixed with new meetings prompt by @zereraz in #218
• Enhancement: Tests for vespa yql generation by @junaid-shirur in #219
• New Feature: GitHub Actions workflow for Bun tests by @junaid-shirur in #204
• New Feature: Add role based checking in UI for integrations page by @kalpadhwaryu in #221
• Bug Fix: Multi line answer by AI caused partial json parse to return empty object causing chat to break by @zereraz in #222
• New Feature: Unified deployment - a single command to run our app @oindrila-b in #230
• Enhancement: Add more file types in dockerignore by @zereraz and @oindrila-b in #232 and #227
• Bug Fix: Calendar sync failed due to change in error message in VespaClient by @kalpadhwaryu in #234

Full Changelog: v0.2.3...v0.2.4

v0.2.3

This fixes our search being broken. Search broke due to error message having special id that we were checking against.
This is risky as one change broke the search so in future we will have to find another way. For now I'm reverting it.

What's Changed (auto generated)

• fix search broken due to error removal from getDocument in vespaClient @zereraz
• feat(refactor): Use schema names as strongly-typed references instead of plain strings by @junaid-shirur in #201

Full Changelog: v0.2.2...v0.2.3

v0.2.2

Bunch of bug fixes, some serious ones for syncing and some improving the quality of life during development.

What's Changed (autogenerated)

• fix(errors): add error as first argument for logger.error by @zereraz in #196
• fix(sync-token): even if contacts sync token is expired we don't exit syncing of the whole workspace by @zereraz in #199
• fix(sync): opposite order of arguments were sent in GetDocument calls in sync by @zereraz in #200

Full Changelog: v0.2.1...v0.2.2

v0.2.1

• chat history pagination
• mainly fixes for citations
• new vespa client as refactoring
• tests for the vespa client
• added tests for citations
• improved deployment, scripts that capture our learnings from deploying on ec2
• error component for all the routes

What's Changed (autogenerated)

• feat(refactor): Modularize AI providers by @junaid-shirur in #182
• update(frontend): frontend packages update to latest versions by @zereraz in #185
• feat(chat-history): Add pagination in chat history list by @kalpadhwaryu in #187
• feat(vespa): Introduce minimal Vespa-Client by @junaid-shirur in #189
• update(server): upgrade package versions in server by @zereraz in #186
• feat(dx-improve): Added scripts to improve deployment by @oindrila-b in #188
• fix(tabs-order): add tabs enum and change the order of ask and search tabs by @zereraz in #190
• feat(errors): add error component and unify the citation regex by @zereraz in #191
• fix(citations): [NaN] if index not in citationMap and add tests by @zereraz in #192
• fix(citation): array of citations now won't be stored by parsed correctly by @zereraz in #194
• update version to 0.2.1 by @zereraz in #195

Full Changelog: v0.2.0...v0.2.1

v0.2.0

Better error handing and retry support.
Added docs(still needs a revamp) and refactored ai directory

What's Changed (autogenerated)

• feat(deploy): Added changes for deployment by @oindrila-b in #172
• feat(deploy): Update vespa deployment script to handle model changes by @junaid-shirur in #174
• feat(chat): Disable sending message if streaming is on by @kalpadhwaryu in #175
• feat(retry): Prevent concurrent retries and disable retries during streaming by @kalpadhwaryu in #176
• feat(dx-improve): Modified code for improved dx by @oindrila-b in #177
• feat(conversation): better support for the conversation and rag system by @zereraz in #178
• fix(chat-error): Make chat usable after error occurs by @kalpadhwaryu in #153
• Add initial set of docs to main by @zereraz in #179
• feat(refactor): Separate types and mappers into different files by @junaid-shirur in #180

Full Changelog: v0.1.0...v0.2.0

v0.1.0

First release:

• We have a working Search and Chat over the Google workspace
• Dockerfile and docker compose
• Service account integration as well as individual OAuth integration

What's Changed (autogenerated)

• Add hybrid search over the chunks using multi vector indexing by @zereraz in #2
• Add websocket support and realtime updates of connector by @zereraz in #3
• Add login with google, authentication and authorization by @zereraz in #6
• Add docker compose for app, vespa and postgres by @zereraz in #8
• Add some basic types to prevent type errors during frontend build by @zereraz in #9
• Insert data of all users in one go by @zereraz in #10
• Restructure the project: add server folder and deployment folder by @zereraz in #12
• Allow OAuth for Google by @zereraz in #21
• Sync Engine by @zereraz in #28
• feat(middleware): add AuthRedirect and redirect unauthorized to /auth by @kalpadhwaryu in #27
• feat(bun): configure bun lockfile to work with git diff by @zereraz in #40
• feat(api): Add /me route to get user and workspace info by @kalpadhwaryu in #46
• feat(eval): FiQA evaluation for search by @junaid-shirur in #47
• feat(logger): Add Structured Logging using pino by @oindrila-b in #48
• feature(search): People search by @zereraz in #45
• chore(type-errors): remove all typescript errors and improve google docs parsing by @zereraz in #55
• feat(biomejs): Add biomejs for formatting by @kalpadhwaryu in #50
• fix(errors): remove all typescript errors, move logger to it's folder by @zereraz in #56
• feat(scripts): Useful scripts to check errors and format code from root folder by @zereraz in #57
• feat(logger): add elapsed time by @zereraz in #58
• feat(search): Create embeddings within vespa by @junaid-shirur in #59
• feat(data-source): add google contacts with syncing by @zereraz in #63
• fix(search): Remove chunk_embeddings Field for Improved Data Ingestion by @junaid-shirur in #64
• feat(google-drive-pdf): Add pdfs from Google Drive by @kalpadhwaryu in #65
• feat(gmail): Gmail support with syncing by @zereraz in #66
• Fix historyId of gmail connector by @zereraz in #71
• feat(search): Improved search by adding multi-phase ranking by @junaid-shirur in #77
• feat(google-sheets): Add Google Sheets by @kalpadhwaryu in #78
• feat(visibility): Add vespa metrics visibility using prometheus and grafana by @oindrila-b in #83
• feat(answer): Add AI based answers on search results by @zereraz in #76
• fix(drive-pdf): delete pdf on error if already downloaded by @zereraz in #85
• fix(observability): improve grafana legend granularity by @zereraz in #86
• feat(error-handling): Added calling error stack by @oindrila-b in #88
• Add Google Slides by @kalpadhwaryu in #90
• feat(chat): ai chat with workspace by @zereraz in #91
• feat(chat): Add automatic title creation based on the user query by @zereraz in #93
• Fix(answer): styling with show full and less buttons by @zereraz in #94
• fix(ingestion): type key was missing in config for discriminated union by @zereraz in #95
• feat(pagination): add ability to go to next page for various cases including filters by @zereraz in #96
• feat(chat): Chat with internal data by @zereraz in #97
• fix(chat): improve the styles in chat by @zereraz in #99
• fix(ingestion): Fix the concurrency bug in google docs by @zereraz in #100
• fix(vespa): change default model and typo fix by @zereraz in #101
• feat(filters): add last updated at filters by @zereraz in #103
• feat(assets): add drive icons and fix minor bug by @zereraz in #104
• feat(agentic-rag): A smarter RAG pipeline by @zereraz in #106
• feat(retry-message): Add retry to an assistant's response by @zereraz in #107
• fix(citations): fix multiple issues with citations by @zereraz in #108
• fix(ingestion): service account union error by @zereraz in #113
• fix(ingestion): incorrect fetching of workspace for service account ingestion by @zereraz in #114
• feat(new-design): unify the search and chat interfaces by @zereraz in #117
• fix(thinking-msg): assistant message when chat created via query param by @zereraz in #119
• feat(google-calendar): Add Google Calendar Events by @kalpadhwaryu in #116
• feat(citations): Add show and hide ui for sources by @zereraz in #120
• fix(citations): map correctly to the original citations from the one's generated by the llm by @zereraz in #121
• feat(dx-improve): made changes for running prod smoothly by @oindrila-b in #124
• fix(search-ui): reduce the summary text for each result by @zereraz in #122
• fix(cost-usage): openai cost for streaming case wasn't working by @zereraz in #125
• Fix(ingestion): ingestion was taking too long by @zereraz in #126
• fix(env): remove frontend baseurl by @zereraz in #127
• fix(search-params): prevent multi level encoding of params by @zereraz in #128
• feat(timerange): to, from support and ui fixes to the search page by @zereraz in #129
• fix(google-sheets): Add sheetTitle as document title by @kalpadhwaryu in #131
• feat(ai-search-answer): improved ai utility to search and answer by @zereraz in #132
• feat(deployment): Made changes in docker compose file and server/logger/index to remove logger folder code creation by @oindrila-b in #133
• feat(deployment): Added .gitkeep inside logs by @oindrila-b in #134
• fix(event-search): Fixed event ranking by @junaid-shirur in #135
• fix(events): Add defaultStartTime, attendeesEmails into permissions & remove email from eventSchema by @kalpadhwaryu in #136
• feat(search): Added freshness signal for files by @junaid-shirur in #137
• feat(chat): improved reasoning strategy and query router by @zereraz in #139
• fix(rate-limit): add retries with exponential backoff by @zereraz in #140
• feat(ingestion): increase concurrency to 2 for service account by @zereraz in #142
• feat(batch): Add batch calls for gmail and sheets by @zereraz in #143
• feat(batch): add batch calls for gmail and table for checking progress in admin settings by @zereraz in #144
• feat(search): Auto suggesting user's history queries by @junaid-shirur in #115
• fix(memory-leak): reduce leak to ingest 1.4M docs under 5GB RAM and add web workers by @zereraz in #150
• feat(deploy): Made changes in docker-compose, dockerfile and server to incorporate deployment requirements by @oindrila-b in #152
• feat(eval): NanoBeir evaluation by @junaid-shirur in #156
• feat(ai): improved ai pipeline optimized for facts by @zereraz in #157
• feat(chat): Delete chat from chat history by @kalpadhwaryu in #155
• fix(events) get data from one year since ingestion and increase vespa limits by @zereraz in #160
• fix(routes): redirect to / when going to /chat by @zereraz in #162
• fix(search): Ensure search results render on first query on build by @junaid-shirur in #163
• feat(citations): end to end working citations by @zereraz in #168
• feat(chat): Add rename chat title by @kalpadhwaryu in #169
• feat(retry): get retry to work on the new architecture and cleanup...