AWSCredentialsProvider issue when using named profile

[tf401]

This is similar to the closed issue [https://github.com/https://github.com/awslabs/amazon-kinesis-client/issues/178]

I followed the above issued solution but got the following error customising the sample.properties file with

# Users can change the credentials provider the KCL will use to retrieve credentials.
# The DefaultAWSCredentialsProviderChain checks several other providers, which is
# described here:
#http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/DefaultAWSCredentialsProviderChain.html
#AWSCredentialsProvider = DefaultAWSCredentialsProviderChain
AWSCredentialsProvider = com.amazonaws.auth.profile.ProfileCredentialsProvider|my_profile

But I get the following error:
java.lang.RuntimeException: com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain: [com.amazonaws.auth.profile.ProfileCredentialsProvider@7517b663: No AWS profile named 'my_profile']

My named profile config file looks like this:

$ cat ~/.aws/config
[default]
region = eu-north-1
output = json
[profile my_profile]
role_arn = arn:aws:iam::<MY_OTHER_AWS_ACCOUNT_ID>:role/OrganizationAccountAccessRole
source_profile = default

I've tried to rename my profile to:

[default]
region = eu-north-1
output = json
[my_profile]
role_arn = arn:aws:iam::<MY_OTHER_AWS_ACCOUNT_ID>:role/OrganizationAccountAccessRole
source_profile = default

But it results in:
2022-07-12 12:43:52,621 [main] WARN s.a.a.p.internal.ProfileFileReader [NONE] - Ignoring profile 'my_profile' on line 12 because it did not start with 'profile ' and it was not 'default'.

Best guess is that I need to provide path to profile configs but after a lot of searching I found no solution.

It is nothing wrong with my credentials & config files since I can interact with the AWS cli and boto3 without problems.
I also tried to deploy my code to EC2 and using DefaultAWSCredentialsProviderChain with no problem.