Partial support of Apple M1/M2 (via CPU mode)

[b0noI]

example of the run:

(base) ➜  llama git:(main) ✗ torchrun --nproc_per_node 1 example_text_completion.py \
    --ckpt_dir llama-2-7b/ \
    --tokenizer_path tokenizer.model \
    --max_seq_len 128 --max_batch_size 4
[2023-07-22 13:50:01,081] torch.distributed.elastic.multiprocessing.redirects: [WARNING] NOTE: Redirects are currently not supported in Windows or MacOs.
> initializing model parallel with size 1
> initializing ddp with size 1
> initializing pipeline with size 1
Loaded in 13.06 seconds
I believe the meaning of life is
> to be happy. I think that if you are happy you will be a good person. I believe that if you are a good person, you will be happy. I believe that if you are happy, you will be a good person. I believe that if you are a good person, you will be happy. I

==================================

Simply put, the theory of relativity states that
> 1) all laws of physics are the same for all observers, and 2) the speed of light in a vacuum is the same for all observers.
1) All laws of physics are the same for all observers
The theory of relativity says that the laws of physics are the same

==================================

A brief message congratulating the team on the launch:

        Hi everyone,

        I just
>
        <a href="#" class="btn btn-success" target="_blank">launched</a>
        my project <a href="#" class="btn btn-success" target="_blank">@<strong>${project.name}</strong></a>
        on <a href

==================================

Translate English to French:

        sea otter => loutre de mer
        peppermint => menthe poivrée
        plush girafe => girafe peluche
        cheese =>
> fromage
        spaghetti => spaghetti
        shampoo => shampoing
        tire => pneu
        candy => bonbon
        pizza => pizza
        car => voiture
        dog => chien
        cat => chat
        orange

==================================

[goswamig]

+1

[YuriHechter]

AttributeError on this line: 'list' object has no attribute 'to'

[b0noI]

thank you, resolved,

somehow this was only impacting example_chat_completion.py only so I missed it

however for example_chat_completion.py there are assertions that are failing, does not look like it has anything to do with my changes. I can make it work but this will require:

• removing assert
• explicitly adjust tensor shape casting

neither of these changes should be committed (but I still can if needed)

[msaroufim]

This is pretty cool, we should look to land this once it supports mps - cc @malfet who has been looking a lot at llm inference on M1

[franciscoabenza]

What is the expected performance (~ tokens per second) for the weakest option: Macbook Air 2020 8GB? It would be nice if there was a table with the different hardware and model sizes.

[PoOwAa]

Small typo, I recommend to refactor it

Suggested change

	def distrubuted_device():
	def distributed_device():

here
and here