-
Notifications
You must be signed in to change notification settings - Fork 171
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
d8d5998
commit 55b3241
Showing
1 changed file
with
10 additions
and
73 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,15 +1,10 @@ | ||
|  | ||
| [](https://travis-ci.org/leonardiwagner/scrapedin) [](https://coveralls.io/github/leonardiwagner/scrapedin?branch=master) | ||
| [](https://travis-ci.org/leonardiwagner/scrapedin) | ||
| [](https://www.npmjs.com/package/scrapedin) | ||
| ---- | ||
| Scraper for LinkedIn full profile data. Unlike others scrapers, it's working in 2019 with their new website. | ||
|
|
||
| Scraper for LinkedIn full profile data.<br/> | ||
| Unlike others scrapers, it's working in 2019 with their new website. | ||
|
|
||
| Install via npm package manager: `npm i scrapedin` | ||
|
|
||
| ### Check your version! | ||
| We need to update at every LinkedIn change. Please check if you have the latest version. | ||
| - Latest release: [v1.0.8](https://github.com/linkedtales/scrapedin/releases/tag/v1.0.8) (latest 16 jul 2019) | ||
| `npm i scrapedin` | ||
|
|
||
| ### Usage Example: | ||
|
|
||
|
|
@@ -20,73 +15,15 @@ const profileScraper = await scrapedin({ email: '[email protected]', password: 'pas | |
| const profile = await profileScraper('https://www.linkedin.com/in/some-profile/') | ||
| ``` | ||
|
|
||
| ### Documentation: | ||
|
|
||
| - `scrapedin(options)` | ||
| - options *Object*: | ||
| - email: LinkedIn login e-mail *(required)* | ||
| - password: LinkedIn login password *(required)* | ||
| - isHeadless: display browser *(default `false`)* | ||
| - hasToLog: print logs on stdout *(default `false`)* | ||
| - puppeteerArgs: puppeteer [launch options Object](https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#puppeteerlaunchoptions). It's very useful, you can also pass [Chromium parameters](https://peter.sh/experiments/chromium-command-line-switches/) at its `args` property, example: `{ args: ['--no-sandbox'] }` *(default `undefined`)* | ||
| - returns: Promise of *profileScraper* function | ||
|
|
||
| - `profileScraper(url, waitTimeMs = 500)` | ||
| - url *string*: A LinkedIn profile URL | ||
| - waitTimeMs *integer*: milliseconds to wait page load before scraping | ||
| - returns: Promise of *profile* Object | ||
|
|
||
| - `profile` Object: | ||
| ```javascript | ||
| { | ||
| profile: { | ||
| name, headline, location, summary, connections, followers | ||
| }, | ||
| positions:[ | ||
| { title, company, description, date1, date2, | ||
| roles: [{ title, description, date1, date2 }] | ||
| } | ||
| ], | ||
| educations: [ | ||
| { title, degree, date1, date2 } | ||
| ], | ||
| skills: [ | ||
| { title, count } | ||
| ], | ||
| recommendations: [ | ||
| { user, text } | ||
| ], | ||
| recommendationsCount: { | ||
| received, given | ||
| }, | ||
| recommendationsReceived: [ | ||
| { user, text } | ||
| ], | ||
| recommendationsGiven: [ | ||
| { user, text } | ||
| ], | ||
| accomplishments: [ | ||
| { count, title, items } | ||
| ], | ||
| volunteerExperience: { | ||
| title, experience, location, description, date1, date2 | ||
| }, | ||
| peopleAlsoViewed: [ | ||
| { user, text } | ||
| ] | ||
| } | ||
| ``` | ||
| ### Tips | ||
|
|
||
| - We already built a crawler to automatically collect multiple profiles, so check it out: [scrapedin-linkedin-crawler](https://github.com/linkedtales/scrapedin-linkedin-crawler) | ||
| - If you are looking for a crawler to automatically extract multiple profiles see [scrapedin-crawler](https://github.com/linkedtales/scrapedin-linkedin-crawler) | ||
|
|
||
| - Usually in the first run LinkedIn asks for a manual check, to solve that you should: | ||
| - set `isHeadless` to `false` on scrapedin to solve the manual check in the browser. | ||
| - set `waitTimeMs` with a large number (such as `10000`) to you have time to solve the manual check. | ||
| ### Start Guide: | ||
|
|
||
| After doing the manual check once you can go back with `isHeadless` and `waitTimeMs` previous values and start the scraping. | ||
| - [Basic Tutorial](https://github.com/linkedtales/scrapedin/wiki/Basic-Tutorial) | ||
| - [Using Cookies to Login](https://github.com/linkedtales/scrapedin/wiki/Using-Cookies-To-Login) | ||
| - [Tips](https://github.com/linkedtales/scrapedin/wiki/Tips) | ||
| - [Documentation](https://github.com/linkedtales/scrapedin/wiki/Documentation) | ||
|
|
||
| We still don't have a solution for that on remote servers without GUI, if you have any idea [please tell us!](https://github.com/linkedtales/scrapedin/issues) | ||
|
|
||
| ### Contribution | ||
|
|
||
|
|
||