Skip to main content

Numeric overflow in expression idea java

· One min read

背景​

在使用java的idea的时候,有如下代码

Date date = new Date();
date.getTime() + 30 * 24 * 60 * 60 * 1000;

其中30 * 24 * 60 * 60 * 1000 会变成负数,而且idea会提示:

Numeric overflow in expression

改成下面的样子就可以去掉警告:

date.getTime() + 30L * 24 * 60 * 60 * 1000

原理​

原理就是变量提升的步骤和溢出的逻辑交叉在一起

found duplicate key xxx spring boot

· One min read

背景​

项目是springboot 框架 发现下面错误: found duplicate key xxx

Caused by: while constructing a mapping
 in 'reader', line 1, column 1:
    management.server.port: 59326
    ^
found duplicate key cms
 in 'reader', line 286, column 1:
    cms:
    ^

	at org.yaml.snakeyaml.constructor.SafeConstructor.processDuplicateKeys(SafeConstructor.java:106)
	at org.yaml.snakeyaml.constructor.SafeConstructor.flattenMapping(SafeConstructor.java:76)
	at org.yaml.snakeyaml.constructor.SafeConstructor.constructMapping2ndStep(SafeConstructor.java:189)
	at org.yaml.snakeyaml.constructor.BaseConstructor.constructMapping(BaseConstructor.java:460)
	at org.yaml.snakeyaml.constructor.SafeConstructor$ConstructYamlMap.construct(SafeConstructor.java:556)
	at org.yaml.snakeyaml.constructor.BaseConstructor.constructObjectNoCheck(BaseConstructor.java:230)
	at org.yaml.snakeyaml.constructor.BaseConstructor.constructObject(BaseConstructor.java:219)
	at org.springframework.boot.env.OriginTrackedYamlLoader$OriginTrackingConstructor.constructObject(OriginTrackedYamlLoader.java:105)
	at org.yaml.snakeyaml.constructor.BaseConstructor.constructDocument(BaseConstructor.java:173)
	at org.yaml.snakeyaml.constructor.BaseConstructor.getData(BaseConstructor.java:138)
	at org.yaml.snakeyaml.Yaml$1.next(Yaml.java:494)
	at org.springframework.beans.factory.config.YamlProcessor.process(YamlProcessor.java:200)
	at org.springframework.beans.factory.config.YamlProcessor.process(YamlProcessor.java:164)
	at org.springframework.boot.env.OriginTrackedYamlLoader.load(OriginTrackedYamlLoader.java:82)
	at org.springframework.boot.env.YamlPropertySourceLoader.load(YamlPropertySourceLoader.java:50)
	at org.springframework.boot.context.config.ConfigFileApplicationListener$Loader.loadDocuments(ConfigFileApplicationListener.java:632)
	at org.springframework.boot.context.config.ConfigFileApplicationListener$Loader.load(ConfigFileApplicationListener.java:534)
	... 64 more


解决方案​

yml文件上有多个相同的都叫cms的key,所以冲突了,去掉一个即可解决

Unable to make protected final java.lang.Class java.lang.ClassLoader.defineClass

· One min read

背景​

本地编译报错

Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make protected final java.lang.Class java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain) throws java.lang.ClassFormatError accessible: module java.base does not "opens java.lang" to unnamed module @49dc7102

排查​

本地编译的时候,发现报这个错,而同事不会有这个错误.
类加载器相关,第一个怀疑的是jdk,于是最后找到原因是用的jdk版本不对,同事用的是jdk11
我刚刚升级的idea不久,idea使用的是默认的jdk17

原因​

项目使用的是jdk11 ,而我idea使用的是jdk17

修改方式​

进入:

File --> Project Structure

选择jdk11,然后问题解决 图片

milvus 编译使用

· One min read

背景​

# Clone github repository.
$ git clone https://github.com/milvus-io/milvus.git

# Install third-party dependencies.
$ cd milvus/
$ ./scripts/install_deps.sh

# Compile Milvus.
$ make

相关错误​

  • Could NOT find BLAS (missing: BLAS_LIBRARIES) 解决方案
sudo apt-get update
sudo apt-get install -y libopenblas-dev
  • ./milvus: error while loading shared libraries: libtbbmalloc.so.2: cannot open shared object file: No such file or directory 解决方案:
sudo apt-get install libtbb2

java sort default order

· 2 min read

背景​

java 的Array.sort()或者.stream.sorted() 都会使用Comparable<T> 作为参数
目前需要了解这些排序函数究竟是升序还是降序的

一句话答案​

所有的排序都是升序的ascending

原因​

Array.sort() , .stream.sorted() 都使用Comparable<T>这个类,都需要实现接口int compare(T o1, T o2);

我们看看接口compare的注释:

Params:
o1 – the first object to be compared. o2 – the second object to be compared.
Returns:
a negative integer, zero, or a positive integer as the first argument is less than, equal to, or greater than the second.
int compare(T o1, T o2);

入参有两个:第一个参数o1,第二个是o2 返回值:

  • 如果o1 < o2 返回 负数
  • 如果o1 > o2 返回正数
  • 如果o1 = o2,返回0

再看看注释java.util.Comparator<T> , 默认都是natural ordering

    /**
     * Returns a comparator that imposes the reverse of the <em>natural
     * ordering</em>.
     *
     * <p>The returned comparator is serializable and throws {@link
     * NullPointerException} when comparing {@code null}.
     *
     * @param  <T> the {@link Comparable} type of element to be compared
     * @return a comparator that imposes the reverse of the <i>natural
     *         ordering</i> on {@code Comparable} objects.
     * @see Comparable
     * @since 1.8
     */
    public static <T extends Comparable<? super T>> Comparator<T> reverseOrder() {
        return Collections.reverseOrder();
    }


相关阅读​

自然排序说明 stackoverflow

lucene 搜索过程

· 13 min read

背景​

了解lucene的搜索过程:

  • 分词
  • 算每个分词的权重，排序取topk

代码堆栈​

  • 写入过程：
add:473, FSTCompiler (org.apache.lucene.util.fst)
compileIndex:504, Lucene90BlockTreeTermsWriter$PendingBlock (org.apache.lucene.codecs.lucene90.blocktree)
writeBlocks:725, Lucene90BlockTreeTermsWriter$TermsWriter (org.apache.lucene.codecs.lucene90.blocktree)
finish:1105, Lucene90BlockTreeTermsWriter$TermsWriter (org.apache.lucene.codecs.lucene90.blocktree)
write:370, Lucene90BlockTreeTermsWriter (org.apache.lucene.codecs.lucene90.blocktree)
write:172, PerFieldPostingsFormat$FieldsWriter (org.apache.lucene.codecs.perfield)
flush:135, FreqProxTermsWriter (org.apache.lucene.index)
flush:310, IndexingChain (org.apache.lucene.index)
flush:392, DocumentsWriterPerThread (org.apache.lucene.index)
doFlush:492, DocumentsWriter (org.apache.lucene.index)
flushAllThreads:671, DocumentsWriter (org.apache.lucene.index)
doFlush:4194, IndexWriter (org.apache.lucene.index)
flush:4168, IndexWriter (org.apache.lucene.index)
shutdown:1322, IndexWriter (org.apache.lucene.index)
close:1362, IndexWriter (org.apache.lucene.index)
doTestSearch:133, FstTest (com.dinosaur.lucene.demo)
  • 读的过程
findTargetArc:1418, FST (org.apache.lucene.util.fst)
seekExact:511, SegmentTermsEnum (org.apache.lucene.codecs.lucene90.blocktree)
loadTermsEnum:111, TermStates (org.apache.lucene.index)
build:96, TermStates (org.apache.lucene.index)
createWeight:227, TermQuery (org.apache.lucene.search)
createWeight:904, IndexSearcher (org.apache.lucene.search)
search:687, IndexSearcher (org.apache.lucene.search)
searchAfter:523, IndexSearcher (org.apache.lucene.search)
search:538, IndexSearcher (org.apache.lucene.search)
doPagingSearch:158, SearchFiles (com.dinosaur.lucene.demo)
testSearch:128, SearchFiles (com.dinosaur.lucene.demo)

例子​

cfe 文件​

$ hexdump  app/index/_3.cfs
000000  3f d7 6c 17 14 4c 75 63 65 6e 65 39 30 43 6f 6d 
000010  70 6f 75 6e 64 44 61 74 61 00 00 00 00 7a fc 30 
000020  52 e0 51 d2 54 be 49 7f 21 78 69 fe c4 00 00 00 
000030  3f d7 6c 17 11 4c 75 63 65 6e 65 39 30 4e 6f 72 
000040  6d 73 44 61 74 61 00 00 00 00 7a fc 30 52 e0 51 
000050  d2 54 be 49 7f 21 78 69 fe c4 00 04 03 c0 28 93 
000060  e8 00 00 00 00 00 00 00 00 f0 6a f4 62 00 00 00 
000070  3f d7 6c 17 16 4c 75 63 65 6e 65 39 30 46 69 65 
000080  6c 64 73 49 6e 64 65 78 49 64 78 00 00 00 00 7a
000090  fc 30 52 e0 51 d2 54 be 49 7f 21 78 69 fe c4 00
0000a0  c0 28 93 e8 00 00 00 00 00 00 00 00 92 7f 21 bb
0000b0  3f d7 6c 17 19 4c 75 63 65 6e 65 39 30 50 6f 69
0000c0  6e 74 73 46 6f 72 6d 61 74 49 6e 64 65 78 00 00
0000d0  00 00 7a fc 30 52 e0 51 d2 54 be 49 7f 21 78 69
0000e0  fe c4 00 32 c0 28 93 e8 00 00 00 00 00 00 00 00
0000f0  f7 61 6e 2f 00 00 00 00 3f d7 6c 17 13 42 6c 6f
000100  63 6b 54 72 65 65 54 65 72 6d 73 49 6e 64 65 78
000110  00 00 00 00 7a fc 30 52 e0 51 d2 54 be 49 7f 21
000120  78 69 fe c4 0a 4c 75 63 65 6e 65 39 30 5f 30 00
000130  00 c0 28 93 e8 00 00 00 00 00 00 00 00 07 1a 7b
000140  47 00 00 00 00 00 00 00 3f d7 6c 17 18 4c 75 63
000150  65 6e 65 39 30 50 6f 69 6e 74 73 46 6f 72 6d 61
000160  74 44 61 74 61 00 00 00 00 7a fc 30 52 e0 51 d2
000170  54 be 49 7f 21 78 69 fe c4 00 02 fe 00 08 80 00
000180  01 88 d2 0f 28 0d ff c0 28 93 e8 00 00 00 00 00
000190  00 00 00 6d 43 fa 6e 00 3f d7 6c 17 19 4c 75 63
0001a0  65 6e 65 39 30 50 6f 73 74 69 6e 67 73 57 72 69
0001b0  74 65 72 44 6f 63 00 00 00 00 7a fc 30 52 e0 51
0001c0  d2 54 be 49 7f 21 78 69 fe c4 0a 4c 75 63 65 6e
0001d0  65 39 30 5f 30 01 03 01 03 c0 28 93 e8 00 00 00   <---   右边的01 03 是you的两个docid
0001e0  00 00 00 00 00 26 f5 75 88 00 00 00 00 00 00 00
0001f0  3f d7 6c 17 19 4c 75 63 65 6e 65 39 30 50 6f 73
000200  74 69 6e 67 73 57 72 69 74 65 72 50 6f 73 00 00
000210  00 00 7a fc 30 52 e0 51 d2 54 be 49 7f 21 78 69
000220  fe c4 0a 4c 75 63 65 6e 65 39 30 5f 30 02 00 00
000230  01 02 03 01 c0 28 93 e8 00 00 00 00 00 00 00 00
000240  c5 ac 32 b6 00 00 00 00 3f d7 6c 17 15 4c 75 63 
000250  65 6e 65 39 30 4e 6f 72 6d 73 4d 65 74 61 64 61
000260  74 61 00 00 00 00 7a fc 30 52 e0 51 d2 54 be 49
000270  7f 21 78 69 fe c4 00 02 00 00 00 ff ff ff ff ff
000280  ff ff ff 00 00 00 00 00 00 00 00 ff ff ff 02 00
000290  00 00 01 2b 00 00 00 00 00 00 00 ff ff ff ff c0
0002a0  28 93 e8 00 00 00 00 00 00 00 00 1c 85 f4 99 00
0002b0  3f d7 6c 17 1c 4c 75 63 65 6e 65 39 30 53 74 6f
0002c0  72 65 64 46 69 65 6c 64 73 46 61 73 74 44 61 74
0002d0  61 00 00 00 01 7a fc 30 52 e0 51 d2 54 be 49 7f
0002e0  21 78 69 fe c4 00 00 0a 00 01 08 12 13 01 04 02
0002f0  05 05 05 05 05 05 05 05 05 10 00 40 10 2e 2e 5c
000300  40 64 6f 63 73 40 5c 64 65 6d 40 6f 2e 74 78 40
000310  74 00 11 2e 40 2e 5c 64 6f 40 63 73 5c 64 40 65
000320  6d 6f 32 40 2e 74 78 74 c0 28 93 e8 00 00 00 00
000330  00 00 00 00 81 b0 7e 09 3f d7 6c 17 18 4c 75 63
000340  65 6e 65 39 30 50 6f 69 6e 74 73 46 6f 72 6d 61
000350  74 4d 65 74 61 00 00 00 00 7a fc 30 52 e0 51 d2
000360  54 be 49 7f 21 78 69 fe c4 00 01 00 00 00 3f d7
000370  6c 17 03 42 4b 44 00 00 00 09 01 01 80 04 08 01
000380  80 00 01 88 d2 0f 28 0d 80 00 01 88 d2 0f 28 0d
000390  02 02 01 32 00 00 00 00 00 00 00 33 00 00 00 00
0003a0  00 00 00 ff ff ff ff 44 00 00 00 00 00 00 00 4f 
0003b0  00 00 00 00 00 00 00 c0 28 93 e8 00 00 00 00 00
0003c0  00 00 00 02 3e 97 d6 00 3f d7 6c 17 17 4c 75 63
0003d0  65 6e 65 39 30 46 69 65 6c 64 73 49 6e 64 65 78
0003e0  4d 65 74 61 00 00 00 01 7a fc 30 52 e0 51 d2 54
0003f0  be 49 7f 21 78 69 fe c4 00 80 80 05 02 00 00 00
000400  0a 00 00 00 02 00 00 00 30 00 00 00 00 00 00 00
000410  00 00 00 00 00 00 00 00 00 00 00 40 00 00 00 00
000420  00 00 00 00 00 30 00 00 00 00 00 00 00 36 00 00
000430  00 00 00 00 00 00 00 84 42 00 00 00 00 00 00 00
000440  00 00 30 00 00 00 00 00 00 00 78 00 00 00 00 00
000450  00 00 01 01 02 c0 28 93 e8 00 00 00 00 00 00 00
000460  00 c3 23 d0 d6 00 00 00 3f d7 6c 17 12 42 6c 6f    <-------  3f
000470  63 6b 54 72 65 65 54 65 72 6d 73 44 69 63 74 00
000480  00 00 00 7a fc 30 52 e0 51 d2 54 be 49 7f 21 78
000490  69 fe c4 0a 4c 75 63 65 6e 65 39 30 5f 30 0b 9c    <--------
0004a0  01 61 72 65 68 6f 77 6f 6c 64 73 74 75 64 65 6e
0004b0  74 79 6f 75 0a 03 03 03 07 03 05 04 00 05 04 00     <------  05 04 00 05 04  是position
0004c0  0b 7a 3d 04 00 02 01 01 05 01 00 01 05 8c 02 2e      <------- 7a 3d 04  是很多位置信息
0004d0  2e 5c 64 6f 63 73 5c 64 65 6d 6f 2e 74 78 74 2e
0004e0  2e 5c 64 6f 63 73 5c 64 65 6d 6f 32 2e 74 78 74
0004f0  04 10 11 01 03 04 82 01 00 05 c0 28 93 e8 00 00
000500  00 00 00 00 00 00 1a 7f dc 45 00 00 00 00 00 00
000510  3f d7 6c 17 12 42 6c 6f 63 6b 54 72 65 65 54 65
000520  72 6d 73 4d 65 74 61 00 00 00 00 7a fc 30 52 e0
000530  51 d2 54 be 49 7f 21 78 69 fe c4 0a 4c 75 63 65
000540  6e 65 39 30 5f 30 3f d7 6c 17 1b 4c 75 63 65 6e 
000550  65 39 30 50 6f 73 74 69 6e 67 73 57 72 69 74 65
000560  72 54 65 72 6d 73 00 00 00 00 7a fc 30 52 e0 51
000570  d2 54 be 49 7f 21 78 69 fe c4 0a 4c 75 63 65 6e
000580  65 39 30 5f 30 80 01 02 02 05 02 da 01 07 07 02
000590  03 61 72 65 03 79 6f 75 37 3f d7 6c 17 03 46 53
0005a0  54 00 00 00 08 01 03 01 da 02 00 00 01 00 02 02
0005b0  92 03 02 02 10 2e 2e 5c 64 6f 63 73 5c 64 65 6d
0005c0  6f 2e 74 78 74 11 2e 2e 5c 64 6f 63 73 5c 64 65
0005d0  6d 6f 32 2e 74 78 74 38 3f d7 6c 17 03 46 53 54
0005e0  00 00 00 08 01 03 03 92 02 00 00 01 49 00 00 00
0005f0  00 00 00 00 a2 00 00 00 00 00 00 00 c0 28 93 e8
000600  00 00 00 00 00 00 00 00 c9 44 df a8 00 00 00 00
000610  3f d7 6c 17 12 4c 75 63 65 6e 65 39 34 46 69 65
000620  6c 64 49 6e 66 6f 73 00 00 00 00 7a fc 30 52 e0
000630  51 d2 54 be 49 7f 21 78 69 fe c4 00 03 04 70 61
000640  74 68 00 02 01 00 ff ff ff ff ff ff ff ff 02 1d
000650  50 65 72 46 69 65 6c 64 50 6f 73 74 69 6e 67 73
000660  46 6f 72 6d 61 74 2e 66 6f 72 6d 61 74 08 4c 75
000670  63 65 6e 65 39 30 1d 50 65 72 46 69 65 6c 64 50
000680  6f 73 74 69 6e 67 73 46 6f 72 6d 61 74 2e 73 75
000690  66 66 69 78 01 30 00 00 01 00 08 6d 6f 64 69 66
0006a0  69 65 64 01 00 00 00 ff ff ff ff ff ff ff ff 00
0006b0  01 01 08 00 01 00 08 63 6f 6e 74 65 6e 74 73 02
0006c0  00 03 00 ff ff ff ff ff ff ff ff 02 1d 50 65 72
0006d0  46 69 65 6c 64 50 6f 73 74 69 6e 67 73 46 6f 72
0006e0  6d 61 74 2e 66 6f 72 6d 61 74 08 4c 75 63 65 6e
0006f0  65 39 30 1d 50 65 72 46 69 65 6c 64 50 6f 73 74
000700  69 6e 67 73 46 6f 72 6d 61 74 2e 73 75 66 66 69
000710  78 01 30 00 00 01 00 c0 28 93 e8 00 00 00 00 00
000720  00 00 00 36 55 24 d2 c0 28 93 e8 00 00 00 00 00
000730  00 00 00 41 6a 49 d4

tim文件的偏移是offset=1128 tim 文件

score:250, BM25Similarity$BM25Scorer (org.apache.lucene.search.similarities)
score:60, LeafSimScorer (org.apache.lucene.search)
score:75, TermScorer (org.apache.lucene.search)
collect:73, TopScoreDocCollector$SimpleTopScoreDocCollector$1 (org.apache.lucene.search)
scoreAll:305, Weight$DefaultBulkScorer (org.apache.lucene.search)
score:247, Weight$DefaultBulkScorer (org.apache.lucene.search)
score:38, BulkScorer (org.apache.lucene.search)
search:776, IndexSearcher (org.apache.lucene.search)
search:694, IndexSearcher (org.apache.lucene.search)
search:688, IndexSearcher (org.apache.lucene.search)
searchAfter:523, IndexSearcher (org.apache.lucene.search)
search:538, IndexSearcher (org.apache.lucene.search)
doPagingSearch:161, SearchFiles (com.dinosaur.lucene.skiptest)

readField:248, Lucene90CompressingStoredFieldsReader (org.apache.lucene.codecs.lucene90.compressing)
document:642, Lucene90CompressingStoredFieldsReader (org.apache.lucene.codecs.lucene90.compressing)
document:253, SegmentReader (org.apache.lucene.index)
document:171, BaseCompositeReader (org.apache.lucene.index)
document:411, IndexReader (org.apache.lucene.index)
doc:390, IndexSearcher (org.apache.lucene.search)
doPagingSearch:195, SearchFiles (com.dinosaur.lucene.skiptest)

tim/tip/doc 关系​

tip 是描述一个term的指针 tim 包含term的统计信息 doc 描述的是term对应的docId

也就是说 tip -> tim -> doc

  • 通过tip判断term是否存在
  • 然后通过tip找到tim获取统计信息
  • 然后通过doc 获取包含该term的docId的数组

doc file​

  • doc file open:
<init>:74, Lucene90PostingsReader (org.apache.lucene.codecs.lucene90)
fieldsProducer:424, Lucene90PostingsFormat (org.apache.lucene.codecs.lucene90)
<init>:330, PerFieldPostingsFormat$FieldsReader (org.apache.lucene.codecs.perfield)
fieldsProducer:392, PerFieldPostingsFormat (org.apache.lucene.codecs.perfield)
<init>:118, SegmentCoreReaders (org.apache.lucene.index)
<init>:92, SegmentReader (org.apache.lucene.index)
doBody:94, StandardDirectoryReader$1 (org.apache.lucene.index)
doBody:77, StandardDirectoryReader$1 (org.apache.lucene.index)
run:816, SegmentInfos$FindSegmentsFile (org.apache.lucene.index)
open:109, StandardDirectoryReader (org.apache.lucene.index)
open:67, StandardDirectoryReader (org.apache.lucene.index)
open:60, DirectoryReader (org.apache.lucene.index)
doSearchDemo:25, SimpleSearchTest (com.dinosaur.lucene.demo)

how to find the docId list

org/apache/lucene/codecs/lucene90/Lucene90PostingsReader.java

  final class BlockDocsEnum extends PostingsEnum {

    ...

    public PostingsEnum reset(IntBlockTermState termState, int flags) throws IOException {
      docFreq = termState.docFreq;
      totalTermFreq = indexHasFreq ? termState.totalTermFreq : docFreq;
      docTermStartFP = termState.docStartFP;
      skipOffset = termState.skipOffset;
      singletonDocID = termState.singletonDocID;
      if (docFreq > 1) {
        if (docIn == null) {
          // lazy init
          docIn = startDocIn.clone();
        }
        docIn.seek(docTermStartFP);
      }

      doc = -1;
      this.needsFreq = PostingsEnum.featureRequested(flags, PostingsEnum.FREQS);
      this.isFreqsRead = true;
      if (indexHasFreq == false || needsFreq == false) {
        for (int i = 0; i < ForUtil.BLOCK_SIZE; ++i) {
          freqBuffer[i] = 1;
        }
      }
      accum = 0;
      blockUpto = 0;
      nextSkipDoc = BLOCK_SIZE - 1; // we won't skip if target is found in first block
      docBufferUpto = BLOCK_SIZE;
      skipped = false;
      return this;
    }
}

相关阅读​

nacos client and serve

· 2 min read

背景​

了解nacos 注册/发现/协议

java-nacos-client​

整个堆栈大概是到这里

write:35, NettyWritableBuffer (com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.grpc.netty)
writeRaw:290, MessageFramer (com.alibaba.nacos.shaded.io.grpc.internal)
writeKnownLengthUncompressed:229, MessageFramer (com.alibaba.nacos.shaded.io.grpc.internal)
writeUncompressed:168, MessageFramer (com.alibaba.nacos.shaded.io.grpc.internal)
writePayload:141, MessageFramer (com.alibaba.nacos.shaded.io.grpc.internal)
writeMessage:53, AbstractStream (com.alibaba.nacos.shaded.io.grpc.internal)
writeMessage:37, ForwardingClientStream (com.alibaba.nacos.shaded.io.grpc.internal)
sendMessageInternal:473, ClientCallImpl (com.alibaba.nacos.shaded.io.grpc.internal)
sendMessage:457, ClientCallImpl (com.alibaba.nacos.shaded.io.grpc.internal)
sendMessage:37, ForwardingClientCall (com.alibaba.nacos.shaded.io.grpc)
sendMessage:37, ForwardingClientCall (com.alibaba.nacos.shaded.io.grpc)
asyncUnaryRequestCall:284, ClientCalls (com.alibaba.nacos.shaded.io.grpc.stub)
futureUnaryCall:191, ClientCalls (com.alibaba.nacos.shaded.io.grpc.stub)
request:212, RequestGrpc$RequestFutureStub (com.alibaba.nacos.api.grpc.auto)
request:73, GrpcConnection (com.alibaba.nacos.common.remote.client.grpc)
request:657, RpcClient (com.alibaba.nacos.common.remote.client)
requestToServer:269, NamingGrpcClientProxy (com.alibaba.nacos.client.naming.remote.gprc)
queryInstancesOfService:169, NamingGrpcClientProxy (com.alibaba.nacos.client.naming.remote.gprc)
queryInstancesOfService:111, NamingClientProxyDelegate (com.alibaba.nacos.client.naming.remote)
run:182, ServiceInfoUpdateService$UpdateTask (com.alibaba.nacos.client.naming.core)
call:515, Executors$RunnableAdapter (java.util.concurrent)
run$$$capture:264, FutureTask (java.util.concurrent)
run:-1, FutureTask (java.util.concurrent)
 - Async stack trace
<init>:151, FutureTask (java.util.concurrent)
<init>:215, ScheduledThreadPoolExecutor$ScheduledFutureTask (java.util.concurrent)
schedule:561, ScheduledThreadPoolExecutor (java.util.concurrent)
run:197, ServiceInfoUpdateService$UpdateTask (com.alibaba.nacos.client.naming.core)
call:515, Executors$RunnableAdapter (java.util.concurrent)
run$$$capture:264, FutureTask (java.util.concurrent)
run:-1, FutureTask (java.util.concurrent)
 - Async stack trace
<init>:151, FutureTask (java.util.concurrent)
<init>:215, ScheduledThreadPoolExecutor$ScheduledFutureTask (java.util.concurrent)
schedule:561, ScheduledThreadPoolExecutor (java.util.concurrent)
run:197, ServiceInfoUpdateService$UpdateTask (com.alibaba.nacos.client.naming.core)
call:515, Executors$RunnableAdapter (java.util.concurrent)
run$$$capture:264, FutureTask (java.util.concurrent)
run:-1, FutureTask (java.util.concurrent)
 - Async stack trace
<init>:151, FutureTask (java.util.concurrent)
<init>:215, ScheduledThreadPoolExecutor$ScheduledFutureTask (java.util.concurrent)
schedule:561, ScheduledThreadPoolExecutor (java.util.concurrent)
run:197, ServiceInfoUpdateService$UpdateTask (com.alibaba.nacos.client.naming.core)
call:515, Executors$RunnableAdapter (java.util.concurrent)
run$$$capture:264, FutureTask (java.util.concurrent)
run:-1, FutureTask (java.util.concurrent)
 - Async stack trace
<init>:151, FutureTask (java.util.concurrent)
<init>:215, ScheduledThreadPoolExecutor$ScheduledFutureTask (java.util.concurrent)
schedule:561, ScheduledThreadPoolExecutor (java.util.concurrent)
run:197, ServiceInfoUpdateService$UpdateTask (com.alibaba.nacos.client.naming.core)
call:515, Executors$RunnableAdapter (java.util.concurrent)
run$$$capture:264, FutureTask (java.util.concurrent)
run:-1, FutureTask (java.util.concurrent)
 - Async stack trace
<init>:151, FutureTask (java.util.concurrent)
<init>:215, ScheduledThreadPoolExecutor$ScheduledFutureTask (java.util.concurrent)
schedule:561, ScheduledThreadPoolExecutor (java.util.concurrent)
run:197, ServiceInfoUpdateService$UpdateTask (com.alibaba.nacos.client.naming.core)
call:515, Executors$RunnableAdapter (java.util.concurrent)
run$$$capture:264, FutureTask (java.util.concurrent)
run:-1, FutureTask (java.util.concurrent)
 - Async stack trace
<init>:151, FutureTask (java.util.concurrent)
<init>:215, ScheduledThreadPoolExecutor$ScheduledFutureTask (java.util.concurrent)
schedule:561, ScheduledThreadPoolExecutor (java.util.concurrent)
addTask:104, ServiceInfoUpdateService (com.alibaba.nacos.client.naming.core)
scheduleUpdateIfAbsent:98, ServiceInfoUpdateService (com.alibaba.nacos.client.naming.core)
subscribe:144, NamingClientProxyDelegate (com.alibaba.nacos.client.naming.remote)
subscribe:393, NacosNamingService (com.alibaba.nacos.client.naming)
start:134, NacosWatch (com.alibaba.cloud.nacos.discovery)
doStart:182, DefaultLifecycleProcessor (org.springframework.context.support)
access$200:53, DefaultLifecycleProcessor (org.springframework.context.support)
start:360, DefaultLifecycleProcessor$LifecycleGroup (org.springframework.context.support)
startBeans:158, DefaultLifecycleProcessor (org.springframework.context.support)
onRefresh:122, DefaultLifecycleProcessor (org.springframework.context.support)
finishRefresh:895, AbstractApplicationContext (org.springframework.context.support)
refresh:554, AbstractApplicationContext (org.springframework.context.support)
refresh:143, ServletWebServerApplicationContext (org.springframework.boot.web.servlet.context)
refresh:755, SpringApplication (org.springframework.boot)
refresh:747, SpringApplication (org.springframework.boot)
refreshContext:402, SpringApplication (org.springframework.boot)
run:312, SpringApplication (org.springframework.boot)
main:25, Application (com.patpat.mms)

clickhouse jdbc 1002 error

· 2 min read

背景​

生产环境会有下面错误: Unknown error 1002, server ClickHouseNode , clickhouse使用的是http的协议

Caused by: java.sql.SQLException: Unknown error 1002, server ClickHouseNode(addr=http:xxxx.amazonaws.com:8123, db=personas)@1267316279
	at com.clickhouse.jdbc.SqlExceptionUtils.handle(SqlExceptionUtils.java:54)
	at com.clickhouse.jdbc.SqlExceptionUtils.handle(SqlExceptionUtils.java:69)
	at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.executeStatement(ClickHouseStatementImpl.java:139)
	at com.clickhouse.jdbc.internal.SqlBasedPreparedStatement.executeBatch(SqlBasedPreparedStatement.java:158)
	at com.clickhouse.jdbc.internal.SqlBasedPreparedStatement.execute(SqlBasedPreparedStatement.java:382)
	at com.zaxxer.hikari.pool.ProxyPreparedStatement.execute(ProxyPreparedStatement.java:44)
	at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.execute(HikariProxyPreparedStatement.java)
	at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:47)
	at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:74)
	at org.apache.ibatis.executor.SimpleExecutor.doUpdate(SimpleExecutor.java:50)
	at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:117)
	at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:76)
	at jdk.internal.reflect.GeneratedMethodAccessor202.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:64)
	at com.sun.proxy.$Proxy263.update(Unknown Source)
	at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:194)
	at org.apache.ibatis.session.defaults.DefaultSqlSession.insert(DefaultSqlSession.java:181)
	at jdk.internal.reflect.GeneratedMethodAccessor201.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:427)
	... 18 common frames omitted

翻看源码​

一共有两处会抛出ERROR_UNKNOWN , 代码如下:

// 路径 : com\clickhouse\clickhouse-client\0.3.2\clickhouse-client-0.3.2-sources.jar!\com\clickhouse\client\ClickHouseException.java
    public static final int ERROR_UNKNOWN = 1002;
    private static int extractErrorCode(String errorMessage) {
        if (errorMessage == null || errorMessage.isEmpty()) {
            return ERROR_UNKNOWN;                                     // 抛出unknow
        } else if (errorMessage.startsWith("Poco::Exception. Code: 1000, ")) {
            return ERROR_POCO;
        }

        int startIndex = errorMessage.indexOf(' ');
        if (startIndex >= 0) {
            for (int i = ++startIndex, len = errorMessage.length(); i < len; i++) {
                char ch = errorMessage.charAt(i);
                if (ch == '.' || ch == ',' || Character.isWhitespace(ch)) {
                    try {
                        return Integer.parseInt(errorMessage.substring(startIndex, i));
                    } catch (NumberFormatException e) {
                        // ignore
                    }
                    break;
                }
            }
        }

        // this is confusing as usually it's a client-side exception
        return ERROR_UNKNOWN;                                     // 抛出unknown
    }


看了代码,这个错误外部的异常,不是sql语法错误,所以一般是反向代理一侧有问题,也就是代理层或者clickhouse层有问题

why bison can be find in cmake

· One min read

背景​

我添加cmake的时候,经常看看为什么find_package(bison)可以默认找到

原因​

cmake 有默认的xxx.cmake文件

https://cmake.org/cmake/help/v2.8.9/cmake.html#module:FindBISON

/usr/share/cmake-3.22/Modules/FindBISON.cmake

Invalid JSON text in argument 2 in mysql8

· One min read

背景​

有个update_rules字段是json类型

{"type": "once", "values": []}

想用JSON_CONTAINS提取中间的once内容,发现下面这个sql一直报错:

SELECT  *  FROM  `usergroup`  WHERE JSON_CONTAINS(update_rules , 'once' , '$.type' )

报错一直是Invalid JSON text in argument 2 , 看了很久也没有看出第二个参数哪里错了

mysql官网例子 , 我看官网也是这样

解决方式​

搜索了一下stackoverflow,原来字符串还要在里面加双引号

也就是'once' 改成'"once"'

完整的sql变成如下:

SELECT  *  FROM  `usergroup`  WHERE JSON_CONTAINS(update_rules , '"daily"' , '$.type' )