autocomplete not work

[ysrotciv]

Before submitting your bug report

• I believe this is a bug. I'll try to join the Continue Discord for questions
• I'm not able to find an open issue that reports the same bug
• I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS:macOS
- Continue version:0.0.83
- IDE version:PyCharm 2024.2.4 (Professional Edition)
- Model:deepseek-coder-6.7b-base
- config.json:
  
{
  "models": [
    {
      "title": "deepseek-coder-6.7b-instruct",
      "provider": "sagemaker",
      "region": "XXX",
      "model": "deepseek-ai-deepseek-coder-6-7b-instruc-XXX",
      "profile": "sagemaker",
      "modelArn": "arn:aws:sagemaker:XXX"
    }
  ],
  "slashCommands": [
    {
      "name": "edit",
      "description": "Edit highlighted code"
    },
    {
      "name": "comment",
      "description": "Write comments for the highlighted code"
    },
    {
      "name": "share",
      "description": "Export the current chat session to markdown"
    },
    {
      "name": "cmd",
      "description": "Generate a shell command"
    }
  ],
  "customCommands": [
    {
      "name": "test",
      "prompt": "{{{ input }}}\n\nWrite a comprehensive set of unit tests for the selected code. It should setup, run tests that check for correctness including important edge cases, and teardown. Ensure that the tests are complete and sophisticated. Give the tests just as chat output, don't edit any file.",
      "description": "Write unit tests for highlighted code"
    }
  ],
  "tabAutocompleteModel": {
    "title": "deepseek-coder-6.7b-base",
    "provider": "sagemaker",
    "region": "XXX",
    "model": "deepseek-ai-deepseek-coder-6-7b-base-XXX",
    "profile": "sagemaker",
    "modelArn": "arn:aws:sagemaker:XXX"
  },
  "contextProviders": [
    {
      "name": "diff",
      "params": {}
    },
    {
      "name": "open",
      "params": {}
    },
    {
      "name": "terminal",
      "params": {}
    }
  ]
}

### Description

autocomplete not work
when I type "def add" in pycharm, nothing happened, and two log file show different result
I can find the correct result of autocomplete in prompt.log
But error in core.log

### To reproduce

1.type "def add" in pycharm
2.see error in core.log

### Log output

```Shell
prompt.log:
##### Completion options #####
{
  "contextLength": 200000,
  "model": "deepseek-ai-deepseek-coder-6-7b-base-2024-12-24-07-51-41-384",
  "maxTokens": 4096,
  "temperature": 0.01,
  "stop": [
    "<｜fim▁begin｜>",
    "<｜fim▁hole｜>",
    "<｜fim▁end｜>",
    "//",
    "<｜end▁of▁sentence｜>",
    "/src/",
    "#- coding: utf-8",
    ""
  ],
  "raw": true
}

##### Prompt #####
<｜fim▁begin｜>
# pythonProject/main.py
def add<｜fim▁hole｜><｜fim▁end｜>

Completion:




Completion:
(a, b):
    return a + b


def sub(a, b):
    return a - b

core.log:
Error generating autocompletion:  AbortError: Request aborted

[lkk214]

This issue may have been fixed in #3536, if just completion at the end of file content is not working.

[ysrotciv]

@lkk214 autocompletion works when the cursor is not at the end of file content
but there is still error in core.log
[2025-01-08T04:05:27] Error generating autocompletion: Error: aborted
[2025-01-08T04:05:31] Error generating autocompletion: AbortError: Request aborted
[2025-01-08T04:05:33] Error generating autocompletion: AbortError: Request aborted
[2025-01-08T04:05:38] Error generating autocompletion: AbortError: Request aborted
[2025-01-08T04:05:39] Error generating autocompletion: AbortError: Request aborted

[Patrick-Erichsen]

@ysrotciv are you able to try 0.0.84? You'll need to install from EAP.

There were some other folks having issues with autocomplete in Pycharm on 0.0.83 (e.g. #3577), but it appears to be fixed in 0.0.84

[ysrotciv]

@ysrotciv are you able to try 0.0.84? You'll need to install from EAP.

There were some other folks having issues with autocomplete in Pycharm on 0.0.83 (e.g. #3577), but it appears to be fixed in 0.0.84

After I upgrade continue to 0.0.84, there is still error in core.log and autocomplete works.

[Patrick-Erichsen]

@ysrotciv thanks for installing and reporting back. Same errors in core.log? I'm not really sure what the issue might be at this point unfortunately. @lkk214 , any chance you have additional context here?

[lkk214]

@Patrick-Erichsen I have never encountered this error log. Maybe it is related to the sagemaker provider. I have not used it.
Hi @ysrotciv , I wonder if other providers also have this problem?

[bdavj]

I see the same issue with Bedrock. It sometimes works, but most of the time (even when not at end of file):

[2025-01-15T12:39:26] Error generating autocompletion:  Error: aborted
[2025-01-15T12:39:27] Error generating autocompletion:  AbortError: Request aborted
[2025-01-15T12:39:57] Error generating autocompletion:  AbortError: Request aborted

Continue 0.0.84, IJ 2024.1, Claude 3.5 Sonnet

[Patrick-Erichsen]

@bdavj I believe your specific issue is actually a different problem, see this thread: #3440

We also advise against using Sonnet for autocomplete as it isn't trained for FIM and has high latency - Codestral is a much better option. See here for more info: https://docs.continue.dev/customize/model-types/autocomplete

[bdavj]

@bdavj I believe your specific issue is actually a different problem, see this thread: #3440

We also advise against using Sonnet for autocomplete as it isn't trained for FIM and has high latency - Codestral is a much better option. See here for more info: https://docs.continue.dev/customize/model-types/autocomplete

Many thanks for the pointer. Are there any models within bedrock that are likely to work? (Adding another API/vendor isn't really an option unfortunately)

[Patrick-Erichsen]

@bdavj at the moment no, but Poolside and Mistral both have autocomplete models coming to Bedrock in the neat future. Just updated our docs to mention this: https://github.com/continuedev/continue/pull/3789/files#diff-8c78666b9f1b0d68e9c441d6fa93fa2476e4a6a08426f5b8f082e69830c09939R44