Recognizing ship behavior is important for maritime situation awareness and intelligent transportation management. Some scholars extracted ship behaviors from massive trajectory data by statistical analysis. However, the meaning of the behaviors, i.e., semantic meanings of behaviors and their relationships, are not explicit. Ship behaviors are affected by navigational area and traffic rules, so their meanings can be obtained only in specific maritime situations. The work establishes the semantic model of ship behavior (SMSB) to represent and reason the meaning of the behaviors. Firstly, a semantic network is built based on maritime traffic rules and good seamanship. The corresponding detection methods are then proposed to identify basic ship behaviors in various maritime scenes, including dock, anchorage, traffic lane, and general scenes. After that, dynamic Bayesian network (DBN) is used to reason potential ship behaviors. Finally, trajectory annotation and semantic query of the model are validated in the different scenes of harbor. The basic behaviors and potential behaviors in all typical scenes of any harbor can be obtained accurately and expressed conveniently using the proposed model. The model facilitates the ships behavior research, contributing to the semantic trajectory analysis.@en