Skip to main content

clickhouse 建表

· 2 min read

背景是目前在使用clickhouse，想自己搭建一个实例，并且建表成功

zookeeper​

#### 切换到bin目录
cd apache-zookeeper-3.8.0-bin/bin/
## 启动zk
./zkServer.sh
  • 步骤3：创建zk节点 ： path为/path/to/zookeeper/node
### 启动zkCli
cd apache-zookeeper-3.8.0-bin/bin/
### 启动节点
./zkCli.sh
### 创建zk节点 需要一级一级建
### 创建节点命令： create path
[zk: localhost:2181(CONNECTED) 11] create /path
Created /path
[zk: localhost:2181(CONNECTED) 12] create /path/to
Created /path/to
[zk: localhost:2181(CONNECTED) 13] create /path/to/zookeeper
Created /path/to/zookeeper
[zk: localhost:2181(CONNECTED) 14] create /path/to/zookeeper/node
Created /path/to/zookeeper/node

建表​

建表之前先要把cluseter配置好 我的配置：

<!-- zk 配置 -->
    <zookeeper>
        <node>
            <host>localhost</host>
            <port>2181</port>
        </node>
        <session_timeout_ms>30000</session_timeout_ms>
        <operation_timeout_ms>10000</operation_timeout_ms>
        <!-- Optional. Chroot suffix. Should exist. -->
        <root>/path/to/zookeeper/node</root>

    </zookeeper>
    <!-- 宏变量， 建表的时候指定的path中的变量从宏里面读取 -->
    <macros>
        <cluster>testcluster</cluster>
        <shard>01</shard>
        <replica>example01-01-1</replica>
    </macros>
   <remote_servers>
      <!-- cluster 名称叫做 testcluster ， 名字随便取的-->
    <testcluster>
        <shard>
            <replica>
                <host>localhost</host>
                <port>9000</port>
            </replica>
        </shard>
    </testcluster>
  </remote_servers>
<distributed_ddl>

建表语句：

### 这里的{cluster} 就是上面配置的testcluster
CREATE TABLE test ON CLUSTER `{cluster}`
(
    `timestamp` DateTime,
    `contractid` UInt32,
    `userid` UInt32
)
ENGINE = ReplicatedMergeTree('/clickhouse/tables/{cluster}/{shard}/default/test', '{replica}')
PARTITION BY toYYYYMM(timestamp)
ORDER BY (contractid, toDate(timestamp), userid)
SAMPLE BY userid

Query id: 56c07fac-9a0b-4b0b-bf8f-fb808ce452e6

查询zk配置​

SELECT  path  FROM system.zookeeper

遇到错误​

遇到错误：There is no DistributedDDL configuration in server config

原因是： clickhosue的配置没有配对，需要参考上面给的链接添加配置

<distributed_ddl>
  <!-- Path in ZooKeeper to queue with DDL queries -->
  <path>/clickhouse/task_queue/ddl</path>
  <cleanup_delay_period>60</cleanup_delay_period>
  <task_max_lifetime>86400</task_max_lifetime>
  <max_tasks_in_queue>1000</max_tasks_in_queue>
</distributed_ddl>

clickhosue insert insert_deduplicate

· 3 min read

问题复现​

  • 建表语句如下
CREATE TABLE test ON CLUSTER `{cluster}`
(
    `timestamp` DateTime,
    `contractid` UInt32,
    `userid` UInt32
)
ENGINE = ReplicatedMergeTree('/clickhouse/tables/{cluster}/{shard}/default/test', '{replica}')
PARTITION BY toYYYYMM(timestamp)
ORDER BY (contractid, toDate(timestamp), userid)
SAMPLE BY userid

  • 第一次insert
insert into test ( userid ,contractid ,  timestamp ) values (1,1,'2022-02-02');

  • 返回结果是一行记录：
SELECT *  FROM test;

┌───────────timestamp─┬─contractid─┬─userid─┐
│ 2022-02-02 00:00:00 │          1 │      1 │
└─────────────────────┴────────────┴────────┘

第二次insert

insert into test ( userid ,contractid ,  timestamp ) values (1,1,'2022-02-02');

返回还是一行：

:) insert into test ( userid ,contractid ,  timestamp ) values (1,1,'2022-02-02');

INSERT INTO test (userid, contractid, timestamp) FORMAT Values

Query id: 706e2447-95eb-4515-a7b7-cf363512b673

Ok.

1 row in set. Elapsed: 0.056 sec. 

dai-MS-7B89 :) select * from test

SELECT *
FROM test

Query id: 3ba7cd7f-4621-4286-8646-79737ec3e763

┌───────────timestamp─┬─contractid─┬─userid─┐
│ 2022-02-02 00:00:00 │          1 │      1 │
└─────────────────────┴────────────┴────────┘

1 row in set. Elapsed: 0.030 sec.

两次插入一样的数据的话， clickhouse会做对应的去重操作，这样两次插入只会插入一条数据

如何解决​

clickhouse提供了参数控制是否去重的参数insert-deduplicate

set insert_deduplicate=0;

然后重新insert同一行记录，就不会因为去重导致重复插入数据被丢弃了。

insert into test ( userid ,contractid ,  timestamp ) values (1,1,'2022-02-02');

INSERT INTO test (userid, contractid, timestamp) FORMAT Values

Query id: a8df989b-0b63-4b45-a1b8-22c13b18bf0a

Ok.

1 row in set. Elapsed: 0.070 sec. 

dai-MS-7B89 :) select * from test

SELECT *
FROM test

Query id: e077b55e-bfd9-4678-ae46-9fc05714b3f7

┌───────────timestamp─┬─contractid─┬─userid─┐
│ 2022-02-02 00:00:00 │          1 │      1 │
└─────────────────────┴────────────┴────────┘
┌───────────timestamp─┬─contractid─┬─userid─┐
│ 2022-02-02 00:00:00 │          1 │      1 │
└─────────────────────┴────────────┴────────┘

日志和源码分析​

日志分析​

2022.05.15 23:32:04.515912 [ 68323 ] {64b40d4f-0d00-4747-9af3-4afb56b6a84b} <Trace> MergedBlockOutputStream: filled checksums 202202_2_2_0 (state Temporary)
2022.05.15 23:32:04.517872 [ 68323 ] {64b40d4f-0d00-4747-9af3-4afb56b6a84b} <Debug> default.test (7d656761-7cd0-4866-a43e-f0e4cea97654) (Replicated OutputStream): Wrote block with ID '202202_8166901380224458449_12408515745921908624', 1 rows
2022.05.15 23:32:04.533981 [ 68323 ] {64b40d4f-0d00-4747-9af3-4afb56b6a84b} <Information> default.test (7d656761-7cd0-4866-a43e-f0e4cea97654) (Replicated OutputStream): Block with ID 202202_8166901380224458449_12408515745921908624 already exists locally as part 202202_0_0_0; ignoring it.


用lldb调试clickhouse

lldb ./clickhouse-server
void ReplicatedMergeTreeSink::consume(Chunk chunk)
{
    auto block = getHeader().cloneWithColumns(chunk.detachColumns());

        String block_id;

        if (deduplicate)  // 从上面生成
        {
            String block_dedup_token;  // 生成token

            /// We add the hash from the data and partition identifier to deduplication ID.
            /// That is, do not insert the same data to the same partition twice.

            const String & dedup_token = settings.insert_deduplication_token;
            if (!dedup_token.empty())
            {
                /// multiple blocks can be inserted within the same insert query
                /// an ordinal number is added to dedup token to generate a distinctive block id for each block
                block_dedup_token = fmt::format("{}_{}", dedup_token, chunk_dedup_seqnum);
                ++chunk_dedup_seqnum;
            }

            block_id = temp_part.part->getZeroLevelPartBlockID(block_dedup_token);
            LOG_DEBUG(log, "Wrote block with ID '{}', {} rows", block_id, current_block.block.rows());
        }
        else
        {
            LOG_DEBUG(log, "Wrote block with {} rows", current_block.block.rows());
        }

相关阅读​

java类初始化

· 2 min read
Method* InstanceKlass::class_initializer() const {
  Method* clinit = find_method(
      vmSymbols::class_initializer_name(), vmSymbols::void_method_signature());
  if (clinit != NULL && clinit->has_valid_initializer_flags()) {
    return clinit;
  }
  return NULL;
}

寻找方法：

src\hotspot\share\oops\instanceKlass.cpp

int InstanceKlass::find_method_index(const Array<Method*>* methods,
                                     const Symbol* name,
                                     const Symbol* signature,
                                     OverpassLookupMode overpass_mode,
                                     StaticLookupMode static_mode,
                                     PrivateLookupMode private_mode) {
  const bool skipping_overpass = (overpass_mode == OverpassLookupMode::skip);
  const bool skipping_static = (static_mode == StaticLookupMode::skip);
  const bool skipping_private = (private_mode == PrivateLookupMode::skip);
  const int hit = quick_search(methods, name);
  if (hit != -1) {
    const Method* const m = methods->at(hit);

    // Do linear search to find matching signature.  First, quick check
    // for common case, ignoring overpasses if requested.
    if (method_matches(m, signature, skipping_overpass, skipping_static, skipping_private)) {
      return hit;
    }

    // search downwards through overloaded methods
    int i;
    for (i = hit - 1; i >= 0; --i) {
        const Method* const m = methods->at(i);
        assert(m->is_method(), "must be method");
        if (m->name() != name) {
          break;
        }
        if (method_matches(m, signature, skipping_overpass, skipping_static, skipping_private)) {
          return i;
        }
    }
    // search upwards
    for (i = hit + 1; i < methods->length(); ++i) {
        const Method* const m = methods->at(i);
        assert(m->is_method(), "must be method");
        if (m->name() != name) {
          break;
        }
        if (method_matches(m, signature, skipping_overpass, skipping_static, skipping_private)) {
          return i;
        }
    }
    // not found
#ifdef ASSERT
    const int index = (skipping_overpass || skipping_static || skipping_private) ? -1 :
      linear_search(methods, name, signature);
    assert(-1 == index, "binary search should have found entry %d", index);
#endif
  }
  return -1;
}

相关阅读​

jvm线程实现

· One min read

背景​

了解线程相关处理逻辑

jni​

jni和php的扩展类似,都是用一个符号绑定一个native函数

堆栈​

堆栈:

(gdb) bt
#0  __pthread_create_2_1 (newthread=0x7ffff5aa5458, attr=0x7ffff5aa54a0, start_routine=0x7ffff6c0d1aa <thread_native_entry(Thread*)>, arg=0x7ffff02e7810) at pthread_create.c:625
#1  0x00007ffff6c0d813 in os::create_thread (thread=0x7ffff02e7810, thr_type=os::java_thread, req_stack_size=0) at /home/ubuntu/daixiao/jdk/src/hotspot/os/linux/os_linux.cpp:867
#2  0x00007ffff6ee1eb7 in JavaThread::JavaThread (this=0x7ffff02e7810, entry_point=0x7ffff6837419 <thread_entry(JavaThread*, JavaThread*)>, stack_sz=0) at /home/ubuntu/daixiao/jdk/src/hotspot/share/runtime/thread.cpp:1195
#3  0x00007ffff68375e4 in JVM_StartThread (env=0x7ffff0028c38, jthread=0x7ffff5aa5760) at /home/ubuntu/daixiao/jdk/src/hotspot/share/prims/jvm.cpp:2890
#4  0x00007fffe100f68b in ?? ()
#5  0x00007ffff5aa56e0 in ?? ()
#6  0x00007ffff5aa56f8 in ?? ()
#7  0x0000000000000000 in ?? ()

相关阅读​

dubbo-rpc

· 4 min read
doInvoke:92, DubboInvoker (org.apache.dubbo.rpc.protocol.dubbo)
invoke:173, AbstractInvoker (org.apache.dubbo.rpc.protocol)
invoke:52, AsyncToSyncInvoker (org.apache.dubbo.rpc.protocol)
invoke:78, ListenerInvokerWrapper (org.apache.dubbo.rpc.listener)
invoke:91, MonitorFilter (org.apache.dubbo.monitor.support)
invoke:61, FilterNode (org.apache.dubbo.rpc.protocol)
invoke:52, FutureFilter (org.apache.dubbo.rpc.protocol.dubbo.filter)
invoke:61, FilterNode (org.apache.dubbo.rpc.protocol)
invoke:69, ConsumerContextFilter (org.apache.dubbo.rpc.filter)
invoke:61, FilterNode (org.apache.dubbo.rpc.protocol)
invoke:56, InvokerWrapper (org.apache.dubbo.rpc.protocol)
doInvoke:79, FailoverClusterInvoker (org.apache.dubbo.rpc.cluster.support)
invoke:265, AbstractClusterInvoker (org.apache.dubbo.rpc.cluster.support)
intercept:47, ClusterInterceptor (org.apache.dubbo.rpc.cluster.interceptor)
invoke:92, AbstractCluster$InterceptorInvokerNode (org.apache.dubbo.rpc.cluster.support.wrapper)
invoke:93, MockClusterInvoker (org.apache.dubbo.rpc.cluster.support.wrapper)
invoke:170, MigrationInvoker (org.apache.dubbo.registry.client.migration)
invoke:96, InvokerInvocationHandler (org.apache.dubbo.rpc.proxy)
testRpc:-1, proxy1 (org.apache.dubbo.common.bytecode)
testRpc:30, ThirdDubboManagerImpl (com.patpat.mms.mdp.biz.engine.dependencies.api.impl)
testRpc:41, PushTest (com.patpat.mms.mdp.biz.engine.rest)
invoke0:-1, NativeMethodAccessorImpl (jdk.internal.reflect)
invoke:62, NativeMethodAccessorImpl (jdk.internal.reflect)
invoke:43, DelegatingMethodAccessorImpl (jdk.internal.reflect)
invoke:566, Method (java.lang.reflect)
runReflectiveCall:59, FrameworkMethod$1 (org.junit.runners.model)
run:12, ReflectiveCallable (org.junit.internal.runners.model)
invokeExplosively:56, FrameworkMethod (org.junit.runners.model)
evaluate:17, InvokeMethod (org.junit.internal.runners.statements)
evaluate:74, RunBeforeTestExecutionCallbacks (org.springframework.test.context.junit4.statements)
evaluate:84, RunAfterTestExecutionCallbacks (org.springframework.test.context.junit4.statements)
evaluate:75, RunBeforeTestMethodCallbacks (org.springframework.test.context.junit4.statements)
evaluate:86, RunAfterTestMethodCallbacks (org.springframework.test.context.junit4.statements)
evaluate:84, SpringRepeat (org.springframework.test.context.junit4.statements)
runLeaf:366, ParentRunner (org.junit.runners)
runChild:251, SpringJUnit4ClassRunner (org.springframework.test.context.junit4)
runChild:97, SpringJUnit4ClassRunner (org.springframework.test.context.junit4)
run:331, ParentRunner$4 (org.junit.runners)
schedule:79, ParentRunner$1 (org.junit.runners)
runChildren:329, ParentRunner (org.junit.runners)
access$100:66, ParentRunner (org.junit.runners)
evaluate:293, ParentRunner$2 (org.junit.runners)
evaluate:61, RunBeforeTestClassCallbacks (org.springframework.test.context.junit4.statements)
evaluate:70, RunAfterTestClassCallbacks (org.springframework.test.context.junit4.statements)
evaluate:306, ParentRunner$3 (org.junit.runners)
run:413, ParentRunner (org.junit.runners)
run:190, SpringJUnit4ClassRunner (org.springframework.test.context.junit4)
runChild:128, Suite (org.junit.runners)
runChild:27, Suite (org.junit.runners)
run:331, ParentRunner$4 (org.junit.runners)
schedule:79, ParentRunner$1 (org.junit.runners)
runChildren:329, ParentRunner (org.junit.runners)
access$100:66, ParentRunner (org.junit.runners)
evaluate:293, ParentRunner$2 (org.junit.runners)
evaluate:306, ParentRunner$3 (org.junit.runners)
run:413, ParentRunner (org.junit.runners)
run:137, JUnitCore (org.junit.runner)
startRunnerWithArgs:69, JUnit4IdeaTestRunner (com.intellij.junit4)
execute:38, IdeaTestRunner$Repeater$1 (com.intellij.rt.junit)
repeat:11, TestsRepeater (com.intellij.rt.execution.junit)
startRunnerWithArgs:35, IdeaTestRunner$Repeater (com.intellij.rt.junit)
prepareStreamsAndStart:235, JUnitStarter (com.intellij.rt.junit)
main:54, JUnitStarter (com.intellij.rt.junit)

网络io

send:157, NettyChannel (org.apache.dubbo.remoting.transport.netty4)
send:181, AbstractClient (org.apache.dubbo.remoting.transport)
send:53, AbstractPeer (org.apache.dubbo.remoting.transport)
request:137, HeaderExchangeChannel (org.apache.dubbo.remoting.exchange.support.header)
request:95, HeaderExchangeClient (org.apache.dubbo.remoting.exchange.support.header)
request:93, ReferenceCountExchangeClient (org.apache.dubbo.rpc.protocol.dubbo)
doInvoke:108, DubboInvoker (org.apache.dubbo.rpc.protocol.dubbo)
invoke:173, AbstractInvoker (org.apache.dubbo.rpc.protocol)
invoke:52, AsyncToSyncInvoker (org.apache.dubbo.rpc.protocol)
invoke:78, ListenerInvokerWrapper (org.apache.dubbo.rpc.listener)
invoke:91, MonitorFilter (org.apache.dubbo.monitor.support)
invoke:61, FilterNode (org.apache.dubbo.rpc.protocol)
invoke:52, FutureFilter (org.apache.dubbo.rpc.protocol.dubbo.filter)
invoke:61, FilterNode (org.apache.dubbo.rpc.protocol)
invoke:69, ConsumerContextFilter (org.apache.dubbo.rpc.filter)
invoke:61, FilterNode (org.apache.dubbo.rpc.protocol)
invoke:56, InvokerWrapper (org.apache.dubbo.rpc.protocol)
doInvoke:79, FailoverClusterInvoker (org.apache.dubbo.rpc.cluster.support)
invoke:265, AbstractClusterInvoker (org.apache.dubbo.rpc.cluster.support)
intercept:47, ClusterInterceptor (org.apache.dubbo.rpc.cluster.interceptor)
invoke:92, AbstractCluster$InterceptorInvokerNode (org.apache.dubbo.rpc.cluster.support.wrapper)
invoke:93, MockClusterInvoker (org.apache.dubbo.rpc.cluster.support.wrapper)
invoke:170, MigrationInvoker (org.apache.dubbo.registry.client.migration)
invoke:96, InvokerInvocationHandler (org.apache.dubbo.rpc.proxy)
testRpc:-1, proxy1 (org.apache.dubbo.common.bytecode)
testRpc:30, ThirdDubboManagerImpl (com.patpat.mms.mdp.biz.engine.dependencies.api.impl)
testRpc:41, PushTest (com.patpat.mms.mdp.biz.engine.rest)
invoke0:-1, NativeMethodAccessorImpl (jdk.internal.reflect)
invoke:62, NativeMethodAccessorImpl (jdk.internal.reflect)
invoke:43, DelegatingMethodAccessorImpl (jdk.internal.reflect)
invoke:566, Method (java.lang.reflect)
runReflectiveCall:59, FrameworkMethod$1 (org.junit.runners.model)
run:12, ReflectiveCallable (org.junit.internal.runners.model)
invokeExplosively:56, FrameworkMethod (org.junit.runners.model)
evaluate:17, InvokeMethod (org.junit.internal.runners.statements)
evaluate:74, RunBeforeTestExecutionCallbacks (org.springframework.test.context.junit4.statements)
evaluate:84, RunAfterTestExecutionCallbacks (org.springframework.test.context.junit4.statements)
evaluate:75, RunBeforeTestMethodCallbacks (org.springframework.test.context.junit4.statements)
evaluate:86, RunAfterTestMethodCallbacks (org.springframework.test.context.junit4.statements)
evaluate:84, SpringRepeat (org.springframework.test.context.junit4.statements)
runLeaf:366, ParentRunner (org.junit.runners)
runChild:251, SpringJUnit4ClassRunner (org.springframework.test.context.junit4)
runChild:97, SpringJUnit4ClassRunner (org.springframework.test.context.junit4)
run:331, ParentRunner$4 (org.junit.runners)
schedule:79, ParentRunner$1 (org.junit.runners)
runChildren:329, ParentRunner (org.junit.runners)
access$100:66, ParentRunner (org.junit.runners)
evaluate:293, ParentRunner$2 (org.junit.runners)
evaluate:61, RunBeforeTestClassCallbacks (org.springframework.test.context.junit4.statements)
evaluate:70, RunAfterTestClassCallbacks (org.springframework.test.context.junit4.statements)
evaluate:306, ParentRunner$3 (org.junit.runners)
run:413, ParentRunner (org.junit.runners)
run:190, SpringJUnit4ClassRunner (org.springframework.test.context.junit4)
run:137, JUnitCore (org.junit.runner)
startRunnerWithArgs:69, JUnit4IdeaTestRunner (com.intellij.junit4)
execute:38, IdeaTestRunner$Repeater$1 (com.intellij.rt.junit)
repeat:11, TestsRepeater (com.intellij.rt.execution.junit)
startRunnerWithArgs:35, IdeaTestRunner$Repeater (com.intellij.rt.junit)
prepareStreamsAndStart:235, JUnitStarter (com.intellij.rt.junit)
main:54, JUnitStarter (com.intellij.rt.junit)
            final WriteTask task = WriteTask.newInstance(next, m, promise, flush);

堆栈

run:1067, AbstractChannelHandlerContext$WriteTask (io.netty.channel)
safeExecute$$$capture:164, AbstractEventExecutor (io.netty.util.concurrent)
safeExecute:-1, AbstractEventExecutor (io.netty.util.concurrent)
 - Async stack trace
addTask:-1, SingleThreadEventExecutor (io.netty.util.concurrent)
execute:828, SingleThreadEventExecutor (io.netty.util.concurrent)
execute:818, SingleThreadEventExecutor (io.netty.util.concurrent)
safeExecute:989, AbstractChannelHandlerContext (io.netty.channel)
write:796, AbstractChannelHandlerContext (io.netty.channel)
writeAndFlush:758, AbstractChannelHandlerContext (io.netty.channel)
writeAndFlush:808, AbstractChannelHandlerContext (io.netty.channel)
writeAndFlush:1025, DefaultChannelPipeline (io.netty.channel)
writeAndFlush:306, AbstractChannel (io.netty.channel)
send:162, NettyChannel (org.apache.dubbo.remoting.transport.netty4)
send:181, AbstractClient (org.apache.dubbo.remoting.transport)
send:53, AbstractPeer (org.apache.dubbo.remoting.transport)
request:137, HeaderExchangeChannel (org.apache.dubbo.remoting.exchange.support.header)
request:95, HeaderExchangeClient (org.apache.dubbo.remoting.exchange.support.header)
request:93, ReferenceCountExchangeClient (org.apache.dubbo.rpc.protocol.dubbo)
doInvoke:108, DubboInvoker (org.apache.dubbo.rpc.protocol.dubbo)
invoke:173, AbstractInvoker (org.apache.dubbo.rpc.protocol)
invoke:52, AsyncToSyncInvoker (org.apache.dubbo.rpc.protocol)
invoke:78, ListenerInvokerWrapper (org.apache.dubbo.rpc.listener)
invoke:91, MonitorFilter (org.apache.dubbo.monitor.support)
invoke:61, FilterNode (org.apache.dubbo.rpc.protocol)
invoke:52, FutureFilter (org.apache.dubbo.rpc.protocol.dubbo.filter)
invoke:61, FilterNode (org.apache.dubbo.rpc.protocol)
invoke:69, ConsumerContextFilter (org.apache.dubbo.rpc.filter)
invoke:61, FilterNode (org.apache.dubbo.rpc.protocol)
invoke:56, InvokerWrapper (org.apache.dubbo.rpc.protocol)
doInvoke:79, FailoverClusterInvoker (org.apache.dubbo.rpc.cluster.support)
invoke:265, AbstractClusterInvoker (org.apache.dubbo.rpc.cluster.support)
intercept:47, ClusterInterceptor (org.apache.dubbo.rpc.cluster.interceptor)
invoke:92, AbstractCluster$InterceptorInvokerNode (org.apache.dubbo.rpc.cluster.support.wrapper)
invoke:93, MockClusterInvoker (org.apache.dubbo.rpc.cluster.support.wrapper)
invoke:170, MigrationInvoker (org.apache.dubbo.registry.client.migration)
invoke:96, InvokerInvocationHandler (org.apache.dubbo.rpc.proxy)
testRpc:-1, proxy1 (org.apache.dubbo.common.bytecode)
testRpc:30, ThirdDubboManagerImpl (com.patpat.mms.mdp.biz.engine.dependencies.api.impl)
testRpc:41, PushTest (com.patpat.mms.mdp.biz.engine.rest)
invoke0:-2, NativeMethodAccessorImpl (jdk.internal.reflect)
invoke:62, NativeMethodAccessorImpl (jdk.internal.reflect)
invoke:43, DelegatingMethodAccessorImpl (jdk.internal.reflect)
invoke:566, Method (java.lang.reflect)
runReflectiveCall:59, FrameworkMethod$1 (org.junit.runners.model)
run:12, ReflectiveCallable (org.junit.internal.runners.model)
invokeExplosively:56, FrameworkMethod (org.junit.runners.model)
evaluate:17, InvokeMethod (org.junit.internal.runners.statements)
evaluate:74, RunBeforeTestExecutionCallbacks (org.springframework.test.context.junit4.statements)
evaluate:84, RunAfterTestExecutionCallbacks (org.springframework.test.context.junit4.statements)
evaluate:75, RunBeforeTestMethodCallbacks (org.springframework.test.context.junit4.statements)
evaluate:86, RunAfterTestMethodCallbacks (org.springframework.test.context.junit4.statements)
evaluate:84, SpringRepeat (org.springframework.test.context.junit4.statements)
runLeaf:366, ParentRunner (org.junit.runners)
runChild:251, SpringJUnit4ClassRunner (org.springframework.test.context.junit4)
runChild:97, SpringJUnit4ClassRunner (org.springframework.test.context.junit4)
run:331, ParentRunner$4 (org.junit.runners)
schedule:79, ParentRunner$1 (org.junit.runners)
runChildren:329, ParentRunner (org.junit.runners)
access$100:66, ParentRunner (org.junit.runners)
evaluate:293, ParentRunner$2 (org.junit.runners)
evaluate:61, RunBeforeTestClassCallbacks (org.springframework.test.context.junit4.statements)
evaluate:70, RunAfterTestClassCallbacks (org.springframework.test.context.junit4.statements)
evaluate:306, ParentRunner$3 (org.junit.runners)
run:413, ParentRunner (org.junit.runners)
run:190, SpringJUnit4ClassRunner (org.springframework.test.context.junit4)
run:137, JUnitCore (org.junit.runner)
startRunnerWithArgs:69, JUnit4IdeaTestRunner (com.intellij.junit4)
execute:38, IdeaTestRunner$Repeater$1 (com.intellij.rt.junit)
repeat:11, TestsRepeater (com.intellij.rt.execution.junit)
startRunnerWithArgs:35, IdeaTestRunner$Repeater (com.intellij.rt.junit)
prepareStreamsAndStart:235, JUnitStarter (com.intellij.rt.junit)
main:54, JUnitStarter (com.intellij.rt.junit)

相关阅读​

范型检查

· One min read
A ClassCastException is thrown if a cast is found at run time to be impermissible.
Some casts result in an error at compile time. Some casts can be proven, at compile time,
always to be correct at run time. For example, it is always correct to convert a value of a
class type to the type of its superclass; such a cast should require no special action at run
time. Finally, some casts cannot be proven to be either always correct or always i

相关阅读：

lr parser

· 3 min read

文法介绍​

文法 (Grammar) 由四部分组成:$(V_f , V_n , P , S)$

$V_f$: 描述的是终结符

$V_n$: 描述的是非终结符

P : 产生式

S : 开始的产生式

lr(0)​

right : terminal and notermial left : noterminal procduction: left and right configuration: production whith dot successor: set of configuration

build configure set​

the problem is that how to build the configure set :

the configure set is combine with two set : basic set and closure set

  • basic set ：$\lbrace A \rightarrow \omega S.\omega' | A \rightarrow \omega .S\omega' \in P \rbrace$

  • closure set ：$\lbrace A \rightarrow .\omega | A \rightarrow \omega \in P \quad AND \quad B \rightarrow \phi.A\phi' \in Configure \rbrace$

 The basis set consists of all configurations in S having a marker before an s, but with the
marker moved to follow the s;

closure set : {A -> .w | A->w is production}

lr(0)​

lr(0) 特别在于状态机：

  • reduce state : 转换函数的transition 要么 （一个底部 + 0到1个非终结符）
  • read state ： 转换函数的transition 都是终结符

lr(0) 算法​

lr(0) 描述的需要注意以下几个内容：

  • stack : stack 存了两个内容.一个是state 还有一个是$V$ 也就是终结符和非终结符的并集

冲突​

移入-规约冲突

  • 原因: 存在产生式P1和P2 , P1右部是产生式P2右部的前缀

规约-规约冲突:

  • 原因: 存在产生式P1和P2,P1 右部和P2右部有公共后缀

如何解决冲突​

移入-规约冲突解决: FOLLOW(P)没有交集

规约-规约冲突解决: FIRST(t) 没有交集

  • 举例
S->E
E->E+T | T
T->T*F | F
F->(E) | id

这里

E-> T.      (1)
T->T.*F     (2)

这两个移入规约冲突

产生式1 是产生式2的前缀

为了解决冲突,我们引入了slr(1),改进了lr(0)的遇到冲突的时候无法解决的问题

SLR(1)​

相关阅读​

jdk 编译

· 3 min read

前言​

jdk 某个版本之后是自举的，所以编译jdk之前先有一个jdk。 主要参照https://openjdk.java.net/groups/build/doc/building.html

下载jdk​
字节码​

src\hotspot\share\interpreter\bytecodes.cpp

BytecodeInterpreter::run

(gdb) bt
#0  TemplateTable::if_icmp (cc=4294967295) at /home/ubuntu/jdk/src/hotspot/cpu/x86/templateTable_x86.cpp:2381
#1  0x00007ffff70a9519 in Template::generate (this=0x7ffff7d60be0 <TemplateTable::_template_table+5088>, masm=0x7ffff0019760) at /home/ubuntu/jdk/src/hotspot/share/interpreter/templateTable.cpp:63
#2  0x00007ffff709bde8 in TemplateInterpreterGenerator::generate_and_dispatch (this=0x7ffff5c5da40, t=0x7ffff7d60be0 <TemplateTable::_template_table+5088>, tos_out=vtos)
    at /home/ubuntu/jdk/src/hotspot/share/interpreter/templateInterpreterGenerator.cpp:392
#3  0x00007ffff709b934 in TemplateInterpreterGenerator::set_short_entry_points (this=0x7ffff5c5da40, t=0x7ffff7d60be0 <TemplateTable::_template_table+5088>, 
    bep=@0x7ffff5c5d398: 0x7fffe1008a37 "H\211d$\330H\201", <incomplete sequence \354\200>, cep=@0x7ffff5c5d3a0: 0x7fffe1008a37 "H\211d$\330H\201", <incomplete sequence \354\200>, 
    sep=@0x7ffff5c5d3a8: 0x7fffe1008a37 "H\211d$\330H\201", <incomplete sequence \354\200>, aep=@0x7ffff5c5d3b0: 0x7fffe1008a37 "H\211d$\330H\201", <incomplete sequence \354\200>, 
    iep=@0x7ffff5c5d3b8: 0x7fffe1028807 "PSQRH\213M\330H\205\311\017\204", <incomplete sequence \312>, lep=@0x7ffff5c5d3c0: 0x7fffe1008a37 "H\211d$\330H\201", <incomplete sequence \354\200>, 
    fep=@0x7ffff5c5d3c8: 0x7fffe1008a37 "H\211d$\330H\201", <incomplete sequence \354\200>, dep=@0x7ffff5c5d3d0: 0x7fffe1008a37 "H\211d$\330H\201", <incomplete sequence \354\200>, 
    vep=@0x7ffff5c5d3d8: 0x7fffe1028800 "\213\004$H\203\304\bPSQRH\213M\330H\205\311\017\204", <incomplete sequence \312>) at /home/ubuntu/jdk/src/hotspot/share/interpreter/templateInterpreterGenerator.cpp:356
#4  0x00007ffff709b46c in TemplateInterpreterGenerator::set_entry_points (this=0x7ffff5c5da40, code=Bytecodes::_if_icmpeq) at /home/ubuntu/jdk/src/hotspot/share/interpreter/templateInterpreterGenerator.cpp:325
#5  0x00007ffff709b06d in TemplateInterpreterGenerator::set_entry_points_for_all_bytes (this=0x7ffff5c5da40) at /home/ubuntu/jdk/src/hotspot/share/interpreter/templateInterpreterGenerator.cpp:281
#6  0x00007ffff709ac13 in TemplateInterpreterGenerator::generate_all (this=0x7ffff5c5da40) at /home/ubuntu/jdk/src/hotspot/share/interpreter/templateInterpreterGenerator.cpp:234
#7  0x00007ffff70993cb in TemplateInterpreterGenerator::TemplateInterpreterGenerator (this=0x7ffff5c5da40, _code=0x7ffff00a28b0) at /home/ubuntu/jdk/src/hotspot/share/interpreter/templateInterpreterGenerator.cpp:42
#8  0x00007ffff7097e5b in TemplateInterpreter::initialize () at /home/ubuntu/jdk/src/hotspot/share/interpreter/templateInterpreter.cpp:56
#9  0x00007ffff69e06a2 in interpreter_init () at /home/ubuntu/jdk/src/hotspot/share/interpreter/interpreter.cpp:116
#10 0x00007ffff69bbe73 in init_globals () at /home/ubuntu/jdk/src/hotspot/share/runtime/init.cpp:119
#11 0x00007ffff70d59b7 in Threads::create_vm (args=0x7ffff5c5de20, canTryAgain=0x7ffff5c5dd2b) at /home/ubuntu/jdk/src/hotspot/share/runtime/thread.cpp:3728
#12 0x00007ffff6adcd5d in JNI_CreateJavaVM_inner (vm=0x7ffff5c5de78, penv=0x7ffff5c5de80, args=0x7ffff5c5de20) at /home/ubuntu/jdk/src/hotspot/share/prims/jni.cpp:3945
#13 0x00007ffff6add0b1 in JNI_CreateJavaVM (vm=0x7ffff5c5de78, penv=0x7ffff5c5de80, args=0x7ffff5c5de20) at /home/ubuntu/jdk/src/hotspot/share/prims/jni.cpp:4036
#14 0x00007ffff7fba88c in InitializeJVM (pvm=0x7ffff5c5de78, penv=0x7ffff5c5de80, ifn=0x7ffff5c5ded0) at /home/ubuntu/jdk/src/java.base/share/native/libjli/java.c:1527
#15 0x00007ffff7fb7447 in JavaMain (_args=0x7fffffffb040) at /home/ubuntu/jdk/src/java.base/share/native/libjli/java.c:414
#16 0x00007ffff7d7a609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#17 0x00007ffff7ed8293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

类加载​

Thread 2 "java" hit Breakpoint 3, SystemDictionary::load_instance_class (class_name=0x7fffcc2d90f0, class_loader=..., __the_thread__=0x7ffff001b800)
    at /home/dinosaur/jdk12/jdk-jdk-12-25/src/hotspot/share/classfile/systemDictionary.cpp:1386
1386	InstanceKlass* SystemDictionary::load_instance_class(Symbol* class_name, Handle class_loader, TRAPS) {
(gdb) bt
#0  SystemDictionary::load_instance_class (class_name=0x7fffcc2d90f0, class_loader=..., __the_thread__=0x7ffff001b800)
    at /home/dinosaur/jdk12/jdk-jdk-12-25/src/hotspot/share/classfile/systemDictionary.cpp:1386
#1  0x00007ffff61bf8ab in SystemDictionary::resolve_instance_class_or_null (name=0x7fffcc2d90f0, class_loader=..., protection_domain=..., __the_thread__=0x7ffff001b800)
    at /home/dinosaur/jdk12/jdk-jdk-12-25/src/hotspot/share/classfile/systemDictionary.cpp:854
#2  0x00007ffff61bdcf8 in SystemDictionary::resolve_instance_class_or_null_helper (class_name=0x7fffcc2d90f0, class_loader=..., protection_domain=..., __the_thread__=0x7ffff001b800)
    at /home/dinosaur/jdk12/jdk-jdk-12-25/src/hotspot/share/classfile/systemDictionary.cpp:272
#3  0x00007ffff61bdb5e in SystemDictionary::resolve_or_null (class_name=0x7fffcc2d90f0, class_loader=..., protection_domain=..., __the_thread__=0x7ffff001b800)
    at /home/dinosaur/jdk12/jdk-jdk-12-25/src/hotspot/share/classfile/systemDictionary.cpp:255
#4  0x00007ffff61bd7d1 in SystemDictionary::resolve_or_fail (class_name=0x7fffcc2d90f0, class_loader=..., protection_domain=..., throw_error=true, __the_thread__=0x7ffff001b800)
    at /home/dinosaur/jdk12/jdk-jdk-12-25/src/hotspot/share/classfile/systemDictionary.cpp:203
#5  0x00007ffff61bdae8 in SystemDictionary::resolve_or_fail (class_name=0x7fffcc2d90f0, throw_error=true, __the_thread__=0x7ffff001b800)
    at /home/dinosaur/jdk12/jdk-jdk-12-25/src/hotspot/share/classfile/systemDictionary.cpp:245
#6  0x00007ffff61c3490 in SystemDictionary::resolve_wk_klass (id=SystemDictionary::Object_klass_knum, __the_thread__=0x7ffff001b800)
    at /home/dinosaur/jdk12/jdk-jdk-12-25/src/hotspot/share/classfile/systemDictionary.cpp:1938
#7  0x00007ffff61c35c5 in SystemDictionary::resolve_wk_klasses_until (limit_id=SystemDictionary::Cloneable_klass_knum, start_id=@0x7ffff7fbc974: SystemDictionary::Object_klass_knum, 
    __the_thread__=0x7ffff001b800) at /home/dinosaur/jdk12/jdk-jdk-12-25/src/hotspot/share/classfile/systemDictionary.cpp:1948
#8  0x00007ffff5c5c46d in SystemDictionary::resolve_wk_klasses_through (end_id=SystemDictionary::Class_klass_knum, start_id=@0x7ffff7fbc974: SystemDictionary::Object_klass_knum, 
    __the_thread__=0x7ffff001b800) at /home/dinosaur/jdk12/jdk-jdk-12-25/src/hotspot/share/classfile/systemDictionary.hpp:391
#9  0x00007ffff61c37a3 in SystemDictionary::resolve_well_known_classes (__the_thread__=0x7ffff001b800) at /home/dinosaur/jdk12/jdk-jdk-12-25/src/hotspot/share/classfile/systemDictionary.cpp:1991
#10 0x00007ffff61c32d8 in SystemDictionary::initialize (__the_thread__=0x7ffff001b800) at /home/dinosaur/jdk12/jdk-jdk-12-25/src/hotspot/share/classfile/systemDictionary.cpp:1898
#11 0x00007ffff623b65c in Universe::genesis (__the_thread__=0x7ffff001b800) at /home/dinosaur/jdk12/jdk-jdk-12-25/src/hotspot/share/memory/universe.cpp:329
#12 0x00007ffff623dd21 in universe2_init () at /home/dinosaur/jdk12/jdk-jdk-12-25/src/hotspot/share/memory/universe.cpp:953
#13 0x00007ffff5a575e7 in init_globals () at /home/dinosaur/jdk12/jdk-jdk-12-25/src/hotspot/share/runtime/init.cpp:125
#14 0x00007ffff620ecbe in Threads::create_vm (args=0x7ffff7fbce20, canTryAgain=0x7ffff7fbcd2b) at /home/dinosaur/jdk12/jdk-jdk-12-25/src/hotspot/share/runtime/thread.cpp:3731
#15 0x00007ffff5b6b598 in JNI_CreateJavaVM_inner (vm=0x7ffff7fbce78, penv=0x7ffff7fbce80, args=0x7ffff7fbce20) at /home/dinosaur/jdk12/jdk-jdk-12-25/src/hotspot/share/prims/jni.cpp:3935
#16 0x00007ffff5b6b8c2 in JNI_CreateJavaVM (vm=0x7ffff7fbce78, penv=0x7ffff7fbce80, args=0x7ffff7fbce20) at /home/dinosaur/jdk12/jdk-jdk-12-25/src/hotspot/share/prims/jni.cpp:4021
#17 0x00007ffff7bc6601 in InitializeJVM (pvm=0x7ffff7fbce78, penv=0x7ffff7fbce80, ifn=0x7ffff7fbced0) at /home/dinosaur/jdk12/jdk-jdk-12-25/src/java.base/share/native/libjli/java.c:1529
#18 0x00007ffff7bc320c in JavaMain (_args=0x7fffffffaa30) at /home/dinosaur/jdk12/jdk-jdk-12-25/src/java.base/share/native/libjli/java.c:414
#19 0x00007ffff71956db in start_thread (arg=0x7ffff7fbd700) at pthread_create.c:463
#20 0x00007ffff78ef61f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95


(gdb) p class_name->_body@10
$14 = {"ja", "va", "/l", "an", "g/", "Ob", "je", "ct", "\361", <incomplete sequence \361>, "\377\377"}

解析文件流​

ClassFileParser::parse_stream

find_transitive_override​

重载

find_transitive_override
update_inherited_vtable

签名​

Method::name_and_sig_as_C_string

相关阅读​