Skip to content

Commit 3c75079

Browse files
casiononeaiceflowerpeacewongbinbinCheng
authored
Optimize sidebar (#619)
* Directory structure adjustment (#570) * [WIP]update engine usage document (#577) * update engine usage document * Optimize the control panel instructions * pic url * control panel dir adjust * update * desc errora * add glossary doc (#574) * add glossary doc * [feat:3871] Linkis website document optimization (#603) * version update * small change * control panel, udf update * add udf pic * add advice configuration * update en doc * update udf,spark * update param conf * update udf, pic * add advice configuration * remove microservice division * [feat-3857]Optimized the Links SDK instructions (#592) * Optimized the Links SDK instructions * Optimized the Links SDK instructions * fix tr error * update show * update show * update table * update table * update trino * update img(en) * update img(en) * update en * build error * update en * update uef en * Added how to register a new microservice document (#584) * Added how to register a new microservice document Co-authored-by: Casion <[email protected]> * optimize sidebar * optimize sidebar * update some docs * update some docs * update table docs * fix url bug * update table * update table * update table * update table Co-authored-by: aiceflower <[email protected]> Co-authored-by: peacewong <[email protected]> Co-authored-by: 成彬彬 <[email protected]>
1 parent 1416878 commit 3c75079

File tree

522 files changed

+9551
-4462
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

522 files changed

+9551
-4462
lines changed

blog/2022-03-20-openlookeng.md

+10-10
Original file line numberDiff line numberDiff line change
@@ -8,15 +8,15 @@ tags: [engine]
88
openLooKeng is an "out of the box" engine that supports in-situ analysis of any data, anywhere, including geographically remote data sources. It provides a global view of all data through a SQL 2003 interface. openLooKeng features high availability, auto-scaling, built-in caching and indexing support, providing the reliability needed for enterprise workloads.
99

1010
openLooKeng is used to support data exploration, ad hoc query and batch processing with near real-time latency of 100+ milliseconds to minutes without moving data. openLooKeng also supports hierarchical deployment, enabling geographically remote openLooKeng clusters to participate in the same query. With its cross-region query plan optimization capabilities, queries involving remote data can achieve near "local" performance.
11-
Linkis implements the OpenLooKeng engine to enable Linkis to have the ability to virtualize data and support the submission of cross-source heterogeneous queries, cross-domain and cross-DC query tasks. As a computing middleware, Linkis can connect more low-level computing and storage components by using OpenLooKeng's connector based on the connectivity capability of Linkis' EngineConn.
11+
Linkis implements the openLooKeng engine to enable Linkis to have the ability to virtualize data and support the submission of cross-source heterogeneous queries, cross-domain and cross-DC query tasks. As a computing middleware, Linkis can connect more low-level computing and storage components by using openLooKeng's connector based on the connectivity capability of Linkis' EngineConn.
1212

1313
## Development implementation
14-
The implementation of OpenLooKeng ec is extended based on the EngineConn Plugin (ECP) of Linkis. Because the OpengLooKeng service supports multiple users to query through the Client, the implementation mode is the implementation mode of the multi-user concurrent engine.
14+
The implementation of openLooKeng ec is extended based on the EngineConn Plugin (ECP) of Linkis. Because the OpengLooKeng service supports multiple users to query through the Client, the implementation mode is the implementation mode of the multi-user concurrent engine.
1515
That is, tasks submitted by multiple users can run in one EC process at the same time, which can greatly reuse EC resources and reduce resource waste. The specific class diagram is as follows:
1616

1717
【Missing picture】
1818

19-
The specific implementation is that OpenLooKengEngineConnExecutor inherits from ConcurrentComputationExecutor, supports multi-user multi-task concurrency, and supports docking to multiple different OpenLooKeng clusters.
19+
The specific implementation is that openLooKengEngineConnExecutor inherits from ConcurrentComputationExecutor, supports multi-user multi-task concurrency, and supports docking to multiple different openLooKeng clusters.
2020
## Architecture
2121
Architecture diagram:
2222
![image](https://user-images.githubusercontent.com/7869972/166736911-c0f50968-3996-40d0-afdf-52b35d4cd71c.png)
@@ -25,15 +25,15 @@ Architecture diagram:
2525
The task flow diagram is as follows:
2626
![image](https://user-images.githubusercontent.com/7869972/166737177-57f8f84a-b16d-44bd-b7cf-a61fc2cc160c.png)
2727

28-
The capabilities based on Linkis and OpenLooKeng can provide the following capabilities:
29-
- 1. The connection capability of the computing middleware layer based on Linkis allows upper-layer application tools to quickly connect to OpenLooKeng, submit tasks, and obtain logs, progress, and results.
30-
- 2. Based on the public service capability of Linkis, it can complete custom variable substitution, UDF management, etc. for OpenLooKeng's sql
28+
The capabilities based on Linkis and openLooKeng can provide the following capabilities:
29+
- 1. The connection capability of the computing middleware layer based on Linkis allows upper-layer application tools to quickly connect to openLooKeng, submit tasks, and obtain logs, progress, and results.
30+
- 2. Based on the public service capability of Linkis, it can complete custom variable substitution, UDF management, etc. for openLooKeng's sql
3131
- 3. Based on the context capability of Linkis, the results of OpengLooKeng can be passed to downstream ECs such as Spark and Hive for query
32-
- 4. Linkis-based resource management and multi-tenancy capabilities can isolate tasks from tenants and use OpenLooKeng resources
32+
- 4. Linkis-based resource management and multi-tenancy capabilities can isolate tasks from tenants and use openLooKeng resources
3333
- 5. Based on OpengLooKeng's connector capability, the upper-layer application tool can complete the task of submitting cross-source heterogeneous query, cross-domain and cross-DC query type, and get a second-level return.
3434

3535
## Follow-up plans
3636
In the future, the two communities will continue to cooperate and plan to launch the following functions:
37-
- 1.Linkis supports OpenLooKeng on Yarn mode
38-
- 2. Linkis has completed the resource management and control of OpenLooKeng, tasks can now be queued by Linkis, and submitted only when resources are sufficient
39-
- 3. Based on the mixed computing ability of OpenLooKeng, the ability of Linkis Orchestrator is optimized to complete the mixed computing ability between ECs in the subsequent plan.
37+
- 1.Linkis supports openLooKeng on Yarn mode
38+
- 2. Linkis has completed the resource management and control of openLooKeng, tasks can now be queued by Linkis, and submitted only when resources are sufficient
39+
- 3. Based on the mixed computing ability of openLooKeng, the ability of Linkis Orchestrator is optimized to complete the mixed computing ability between ECs in the subsequent plan.

community/microservice-division.md

-47
This file was deleted.

discuss/project-hisotry.md

+155
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
## Apache Gobblin
2+
- 孵化时间:2017-02-23 ~ 2021-01-20
3+
- 毕业讨论邮件:https://lists.apache.org/thread/9z79b2648bcxy6pw78bfhvq1jkwy0lox
4+
- 孵化信息: https://incubator.apache.org/projects/gobblin.html
5+
- 反馈的问题:
6+
7+
8+
## Apache Ratis
9+
- 孵化时间:2017-01-03 ~ 2021-02-17
10+
- 毕业讨论邮件: https://lists.apache.org/thread/0x2pbbr0khldfo2t8vp49cnl9oz51wcb
11+
- 孵化信息: https://incubator.apache.org/projects/ratis.html
12+
- 反馈的问题:
13+
- 邮件流量问题 Adjust Github nofications settings to use issues@ mailing list
14+
15+
16+
## Apache Pinot
17+
- 孵化时间:2018-10-17 ~ 2021-07-21
18+
- 毕业讨论邮件: https://lists.apache.org/thread/9y63b4fosbocmsj8qlyzg8j4c6166b54
19+
- 孵化信息: https://incubator.apache.org/projects/pinot.html
20+
- 反馈的问题:
21+
- 文档有一些 ASF 软件的品牌问题 https://github.com/prestodb/presto/issues/16271
22+
One other thing you might want to correct, all of the download links to previous version are broken on your download page. [1] The previous version are automatically archived and can be found here. [2] You may also want to consider not calling the convenience binary an “offical binary release”.
23+
24+
25+
## Apache Hop
26+
- 孵化时间:2020-09-24 ~ 2021-12-16
27+
- 毕业讨论邮件: https://lists.apache.org/thread/jj46blkwrhf0l31srtzo5j5dcv7zsqz3
28+
- 孵化信息: https://incubator.apache.org/projects/hop.html
29+
- 反馈的问题:
30+
- 下载区有几个旧版本;这些应该被删除
31+
- 项目描述太长,需优化相关的措辞
32+
33+
## Apache AGE
34+
- 孵化时间:2020-04-29 ~ 2022-05-18
35+
- 毕业讨论邮件: https://lists.apache.org/thread/jvrtvcw1zb8gf286css7tvol94r3qd68
36+
- 孵化信息: https://incubator.apache.org/projects/age.html
37+
- 反馈的问题:
38+
- 三名 PPMC 成员未订阅私人邮件列表。
39+
- 包括一些 PPMC 成员在内的四名 Committer 没有将他们的 apache id 映射到他们的 GitHub id。他们如何做出贡献?
40+
- 没有人加入PPMC
41+
42+
讨论指出的问题比较多,多次发起了
43+
44+
45+
## Apache YuniKorn
46+
- 孵化时间:2020-01-21 ~ 2022-03-16
47+
- 毕业讨论邮件: https://lists.apache.org/thread/o8y4b3mp21f608745ngqqdhc628qtwxf
48+
- 孵化信息: https://incubator.apache.org/projects/yunikorn.html
49+
- 反馈的问题:
50+
51+
52+
## Apache Doris
53+
- 孵化时间:2018-07-18 ~ 2022-06-15
54+
- 毕业讨论邮件: https://lists.apache.org/thread/nbncjl8fdq0bcjop6l4747c8x9c06q00
55+
- 孵化信息: https://incubator.apache.org/projects/doris.html
56+
- 反馈的问题:
57+
- DorisDB 和 StarRocks的侵权问题
58+
- 百度在中国拥有 Doris 商标,该商标是否已转让给 ASF?
59+
- 与 Apache Impala 的关系
60+
61+
注:涉及比较多的商标和License 问题
62+
63+
- 毕业清单 https://cwiki.apache.org/confluence/display/DORIS/Graduation+Check+List
64+
65+
## Apache MXNet
66+
67+
- 孵化时间:2017-01-23 ~ 2022-06
68+
- 毕业讨论邮件: https://lists.apache.org/thread/3rxzjcmo2y457y6r8ohz1j4qv49joyo6
69+
- 孵化信息: https://incubator.apache.org/projects/mxnet.html
70+
- 反馈的问题:
71+
- 不应该使用 Google Analytics,而是使用 Apache 安装的 Matomo 跟踪器。
72+
- 第三方产品名称问题 3rd party product name listed
73+
74+
- - ICLA签署
75+
https://cwiki.apache.org/confluence/display/MXNET/ICLA+Progress
76+
77+
78+
79+
## Apache ShenYu
80+
- 孵化时间:2021-05-03 ~ 2022-06
81+
- 毕业讨论邮件: https://lists.apache.org/thread/cc2myvso0pbyl39yv26bdmzml8jjvpt2
82+
- 孵化信息: https://incubator.apache.org/projects/shenyu.html
83+
- 反馈的问题:
84+
- 主页描述不够精简 最好用简短而清晰的句子来 说明它在做什么
85+
- 网站上提到了 5 个版本,而不是 4 个,在孵化之前的需要标识出 There's no 2.3.0 version release. And the 2.3.0 document is marked legacy now https://github.com/ apache/incubator-shenyu-website/pull/604
86+
- 文档中名字最好带上 Apache 前缀 https://github.com/apache/incubator-shenyu/pull/3568
87+
- 商标问题 \[Branding Issue] Shenyu is still decleared as a project of dromara community https://lists.apache.org/thread/o9bwzkl3cjbrmzx4n36l63xbv0q112px
88+
89+
90+
## Apache DataSketches
91+
- 孵化时间:2019-03-30 ~ 2020-12-16
92+
- 毕业讨论邮件: https://lists.apache.org/thread/kcpo1ngkmh9q3zgxl5vo64m9m62c45m0
93+
- 孵化信息: https://incubator.apache.org/projects/datasketches.html
94+
- 反馈的问题:
95+
- 如果有重要贡献者没有作为初始贡献者加入 podling,或者如果有其他公司实体可以在代码中主张权利,则 podling 必须从这些个人或公司获得 SGA
96+
97+
98+
99+
100+
101+
102+
1. 之前尝试在Jira上提单申请hub账户 但是反馈坐席满了, 推荐使用Github Docker Registry。  这个是否存在合规性问题(官网文档https://incubator.apache.org/guides/distribution.html#docker 是说:
103+
Artifacts need to be placed in https://hub.docker.com/r/apache/<project>)
104+
105+
2. 另外对于容器镜像构建过程中 的Dockfile文件,使用了基础镜像 如centos:7 ,以及通过如yun install unzip 的安装的基础环境软件包, 或则使用curl/ADD/COPY to a destination inside the Docker image. 这些第三方的工具的使用 是否需要单独在LICENSE中进行 罗列说明 。
106+
107+
108+
1. 我们之前申请hub账号权限, 但是反馈seat 坐席满了, 推荐使用Github Docker Registry。  这个是否符合asf docker 镜像分发要求(官网文档https://incubator.apache.org/guides/distribution.html#docker 提到:
109+
Artifacts need to be placed in https://hub.docker.com/r/apache/<project>)
110+
111+
2. 另外对于构建docker 镜像使用的 Dockfile文件,使用了基础镜像 如centos:7 ,以及通过如yun install unzip 的安装的基础环境软件包, 或则使用 ADD/COPY 添加资源 to a destination inside the Docker image. 这些第三方的工具的使用 是否需要单独在LICENSE文件中进行 罗列说明 。
112+
113+
114+
https://issues.apache.org/jira/browse/INFRA-22785
115+
116+
https://issues.apache.org/jira/browse/INFRA-22785
117+
DockerHub is out of seats. Project is using GitHub Registry instead.
118+
119+
About docker image release and license
120+
121+
We applied for the DockerHub permission before[1], but the feedback DockerHub is out of seats. It is recommended to use the Github Docker Registry.
122+
Does this meet the asf docker image distribution requirements (the official website document[2]mentions:
123+
artifacts need to be placed in https://hub.docker.com/r/apache/<project>)
124+
125+
In addition, for the Dockfile file used to build the docker image[3], use the basic image such as centos,
126+
and the basic environment package installed through such as `yun install unzip`,
127+
or use `ADD/COPY` to add resources to a destination inside the Docker image.
128+
Whether the use of these third-party tools needs to be listed separately in the LICENSE file.
129+
If so, for the toolkit installed by the yum command, it seems that the indirect dependency license cannot be listed well.
130+
131+
132+
1. https://issues.apache.org/jira/browse/INFRA-22785
133+
2. https://incubator.apache.org/guides/distribution.html#docker
134+
3. https://github.com/apache/incubator-linkis/blob/dev-1.3.0/linkis-dist/docker/ldh.Dockerfile
135+
136+
137+
We applied for the Docker Hub permission before[1], but the feedback Docker Hub is out of seats.
138+
It is recommended to use the Github Docker Registry.
139+
Does this meet the ASF docker image distribution requirements (The official website document[2]
140+
mentions:artifacts need to be placed in https://hub.docker.com/r/apache/<project>)
141+
142+
In addition, for the Dockfile file used to build the docker image[3], use the basic image such as CentOS,
143+
and the basic environment package installed through such as `yun install unzip`,
144+
or use `ADD/COPY` to add resources to a destination inside the Docker image.
145+
Whether the use of these third-party tools needs to be listed separately in the LICENSE file.
146+
If so, for the toolkit installed by the yum command, it seems that the indirect dependency license cannot be listed well.
147+
148+
149+
1. https://issues.apache.org/jira/browse/INFRA-22785
150+
2. https://incubator.apache.org/guides/distribution.html#docker
151+
3. https://github.com/apache/incubator-linkis/blob/dev-1.3.0/linkis-dist/docker/ldh.Dockerfile
152+
153+
Best Regards!
154+
Apache Linkis(Incubating)
155+
Chen Xia

docs/about/_category_.json

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{
2+
"label": "About Linkis",
3+
"position": 1.0
4+
}

0 commit comments

Comments
 (0)